2024-04-25T13:08:23.230 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:08:23.230 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466 --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} -- /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/orig.config.yaml 2024-04-25T13:08:23.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:08:23.376 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7673466' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4060 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 4d7908725372ec8e17504b848bfa122dd1d43386 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPP72cLkBUWFsVOqjlr1j/ryPTLtkeju2zvuvEvZ7lNgcsXw3PRlD102Dda+ywF6itLI2Pe0pNKSbdAMAdRtEU= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD4lmyq3NdK2McDRLBKiZER1Oh3rB6xbnVAPzL1v8i19kftxVD3zAPtVE/9ulRz6OpkxASXTAp7lqTQWeasB3o= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLv+CTxhdVUwfvxI7kfEa+OjaVy9jsj2qygh4fIrXpwYfmbNpahww1nQZaKIx7C5P5Pvm96OMskcdlHiKac+5Oc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_12:17:05 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:08:23.376 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa; will attempt to use it 2024-04-25T13:08:23.377 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks 2024-04-25T13:08:23.377 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:08:23.378 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:08:23.404 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T13:08:23.404 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:08:23.405 INFO:teuthology.packaging:ref: None 2024-04-25T13:08:23.405 INFO:teuthology.packaging:tag: None 2024-04-25T13:08:23.405 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T13:08:23.405 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:08:23.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T13:08:23.560 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T13:08:23.562 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:08:23.568 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:08:23.568 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:08:23.589 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:08:23.611 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:08:23.627 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:08:23.650 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.741619', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPP72cLkBUWFsVOqjlr1j/ryPTLtkeju2zvuvEvZ7lNgcsXw3PRlD102Dda+ywF6itLI2Pe0pNKSbdAMAdRtEU='} 2024-04-25T13:08:23.672 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.745324', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD4lmyq3NdK2McDRLBKiZER1Oh3rB6xbnVAPzL1v8i19kftxVD3zAPtVE/9ulRz6OpkxASXTAp7lqTQWeasB3o='} 2024-04-25T13:08:23.697 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.743586', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLv+CTxhdVUwfvxI7kfEa+OjaVy9jsj2qygh4fIrXpwYfmbNpahww1nQZaKIx7C5P5Pvm96OMskcdlHiKac+5Oc='} 2024-04-25T13:08:23.697 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:08:23.705 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T13:08:23.705 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T13:08:23.705 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T13:08:23.706 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:08:23.799 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5f072de0d0>, signals=[15]) 2024-04-25T13:08:23.799 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:08:23.807 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:08:23.807 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:08:23.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:08:23.890 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-25T13:08:23.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:08:23.964 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T13:08:23.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:08:24.036 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:08:24.042 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-25T13:08:24.060 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-25T13:08:24.061 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-25T13:08:24.118 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:08:24.119 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:08:24.120 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-25T13:08:24.137 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-25T13:08:24.156 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-25T13:08:24.156 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-25T13:08:24.214 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:08:24.215 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:08:24.215 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:08:24.215 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:08:24.215 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:08:24.215 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:08:24.215 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-25T13:08:24.237 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-04-25T13:08:24.254 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-04-25T13:08:24.255 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-25T13:08:24.311 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-25T13:08:24.312 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T13:08:24.312 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-25T13:08:24.312 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T13:08:24.312 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T13:08:24.312 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T13:08:24.312 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-25T13:08:24.328 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:08:24.339 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:08:24.345 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:08:24.346 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:08:24.348 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:08:24.350 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:08:24.367 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:08:24.374 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:08:24.374 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:08:24.405 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:08:24.407 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:08:24.661 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:08:24.748 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:08:24.780 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T13:08:24.780 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T13:08:24.780 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T13:08:24.781 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:08:24.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:24.798 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-25T13:08:24.855 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-25T13:08:24.856 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-25T13:08:24.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-25T13:08:27.093 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T13:08:28.969 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-25T13:08:33.345 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T13:08:41.086 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 2.5 kB/s | 16 kB 00:06 2024-04-25T13:08:42.597 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T13:08:47.890 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T13:08:49.826 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T13:08:49.826 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T13:08:49.906 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T13:08:49.908 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-25T13:08:49.908 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T13:08:50.013 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-25T13:08:50.538 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:08:47 PM UTC. 2024-04-25T13:08:50.661 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T13:08:50.680 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T13:08:50.681 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-25T13:08:50.732 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-25T13:08:50.732 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-25T13:08:51.252 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 01:08:47 PM UTC. 2024-04-25T13:08:51.372 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T13:08:51.373 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:51.373 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T13:08:51.373 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T13:08:51.374 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T13:08:51.655 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-25T13:08:51.655 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:08:51.656 INFO:teuthology.orchestra.run.smithi037.stdout:Total 904 kB/s | 255 kB 00:00 2024-04-25T13:08:51.675 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T13:08:51.861 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T13:08:51.861 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T13:08:51.866 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T13:08:51.867 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T13:08:52.004 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T13:08:52.089 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:08:52.090 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:08:52.716 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:08:52.717 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T13:08:53.415 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T13:08:53.415 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:53.416 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-25T13:08:53.416 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T13:08:53.416 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:08:53.416 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T13:08:53.721 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T13:08:53.770 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T13:08:53.770 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T13:08:53.770 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T13:08:53.771 DEBUG:teuthology.parallel:result is None 2024-04-25T13:08:53.771 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:08:53.779 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:08:53.779 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:08:53.781 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:08:53.783 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:08:53.801 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:08:53.815 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:08:53.825 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:08:53.825 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:08:53.836 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:08:53.842 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:08:53.971 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:08:53.978 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:08:53.978 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:08:53.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:53.996 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:08:54.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:54.012 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:08:54.014 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:08:54.039 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:08:54.050 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.070 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.074 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.093 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.100 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.146 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:08:54.148 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:08:54.155 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:08:54.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:08:54.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:08:54.160 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:08:54.185 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:08:54.195 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:08:54.195 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:08:54.223 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:08:54.228 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:08:54.243 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.363 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:54.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:54.487 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:08:54.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:08:54.551 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.653 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:54.738 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:54.780 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:08:54.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:08:54.846 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:08:54.933 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:55.014 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:08:55.057 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:08:55.057 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:08:55.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-25T13:08:55.123 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-25T13:08:55.126 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-04-25T13:08:55.160 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:08:55.162 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:08:55.197 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:08:55.677 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:08:55.685 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:08:55.685 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:08:55.697 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:08:55.707 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T13:08:55.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-25T13:08:55.767 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-25T13:08:55.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-25T13:08:55.835 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-25T13:08:55.837 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-04-25T13:08:55.900 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-04-25T13:08:55.902 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:08:55.902 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-25T13:08:55.929 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-25T13:08:55.929 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-25T13:08:55.966 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-25T13:08:55.966 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-04-25T13:08:56.003 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-04-25T13:08:56.003 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive'} 2024-04-25T13:08:56.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:08:56.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:56.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:08:56.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:56.056 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:08:56.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:08:56.083 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:08:56.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:08:56.108 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:08:56.135 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-04-25T13:08:56.169 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:08:56.179 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T13:08:56.204 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T13:08:56.360 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:08:56.407 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:08:56.408 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi044.front.sepia.ceph.com,smithi069.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:12:55.690 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com')] 2024-04-25T13:12:55.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T13:12:55.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:12:55.800 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-25T13:12:55.886 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T13:12:55.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-25T13:12:55.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:12:55.964 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-25T13:12:56.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-25T13:12:56.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T13:12:56.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:12:56.124 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-25T13:12:56.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-25T13:12:56.205 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:12:56.217 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:12:56.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:12:56.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:12:56.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:12:56.220 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:12:56.223 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:12:56.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:12:56.251 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:12:56.254 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:12:56.267 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:12:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:12:56.274 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T13:12:56.287 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T13:12:56.292 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-25T13:12:56.295 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-25T13:12:56.304 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-25T13:12:56.307 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-25T13:12:56.309 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-04-25T13:12:56.319 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:12:56.319 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-04-25T13:12:56.322 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:12:56.333 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T13:12:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:12:56.336 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:12:56.345 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T13:12:56.392 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:12:56.393 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:12:56.396 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:12:56.396 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T13:12:56.396 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.396 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.397 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.397 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.397 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-25T13:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.398 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.411 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.414 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:12:56.415 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:12:56.424 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:12:56.424 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T13:12:56.424 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T13:12:56.424 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:12:56.430 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:12:56.430 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:12:56.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:12:56.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:12:56.433 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:12:56.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:12:56.434 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T13:12:56.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:12:56.590 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:12:56.590 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:12:56.700 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:12:56.700 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:12:56.700 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:12:56.714 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:12:56.714 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:12:56.715 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T13:12:56.715 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T13:12:56.724 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:12:56.724 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:12:56.788 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:12:56.828 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:12:56.828 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:12:56.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:12:56.856 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:12:56.856 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:12:56.861 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:12:56.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:12:56.924 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-04-25T13:12:56.926 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-04-25T13:12:56.937 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T13:12:56.937 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:12:56.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T13:12:56.964 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T13:12:56.964 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T13:12:56.997 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:12:57.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T13:12:57.062 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-25T13:12:57.063 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-25T13:12:57.108 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T13:12:57.147 INFO:teuthology.orchestra.run.smithi069.stdout:48 files removed 2024-04-25T13:12:57.173 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-25T13:12:57.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:12:57.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-25T13:12:57.297 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-25T13:12:57.324 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:12:57.410 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-25T13:12:57.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T13:12:57.909 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-25T13:12:57.979 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 219 kB/s | 83 kB 00:00 2024-04-25T13:12:58.197 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 173 kB/s | 83 kB 00:00 2024-04-25T13:12:58.301 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T13:12:58.380 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T13:12:58.637 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T13:12:58.831 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-04-25T13:12:58.843 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-25T13:12:59.137 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-04-25T13:13:00.132 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-04-25T13:13:00.204 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-04-25T13:13:00.491 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-25T13:13:02.488 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T13:13:02.541 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T13:13:03.076 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-25T13:13:06.697 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-25T13:13:06.751 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T13:13:07.385 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T13:13:08.420 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-25T13:13:08.557 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-25T13:13:10.258 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 9.6 kB/s | 16 kB 00:01 2024-04-25T13:13:10.641 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-25T13:13:11.142 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-25T13:13:12.746 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-04-25T13:13:15.630 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T13:13:16.367 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:17.781 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:17.792 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:13:17.793 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:13:17.794 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:13:17.795 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:13:17.796 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:13:17.799 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:13:17.860 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:13:17.876 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:13:17.877 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:17.878 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:13:17.879 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:13:17.880 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:13:17.881 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:13:17.882 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:13:17.883 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:13:17.884 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:13:17.885 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:13:17.886 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 4 Packages 2024-04-25T13:13:17.887 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 222 M 2024-04-25T13:13:17.888 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-04-25T13:13:18.043 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T13:13:18.585 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:18.586 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:18.597 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:13:18.598 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:13:18.599 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:13:18.600 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:13:18.601 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:13:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:13:18.604 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:13:18.671 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T13:13:18.680 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:13:18.681 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:13:18.682 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:13:18.683 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:13:18.684 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:13:18.685 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:13:18.686 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:13:18.687 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:13:18.688 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:13:18.689 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:13:18.690 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:13:18.691 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-25T13:13:18.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:13:18.693 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-25T13:13:18.693 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T13:13:20.253 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:20.254 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T13:13:20.266 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T13:13:20.266 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T13:13:20.267 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T13:13:20.267 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T13:13:20.269 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T13:13:20.271 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T13:13:20.272 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T13:13:20.338 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T13:13:20.356 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T13:13:20.357 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T13:13:20.358 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T13:13:20.359 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T13:13:20.360 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T13:13:20.361 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T13:13:20.362 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T13:13:20.363 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T13:13:20.364 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T13:13:20.365 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T13:13:20.366 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T13:13:20.367 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T13:13:20.368 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 4 Packages 2024-04-25T13:13:20.369 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 222 M 2024-04-25T13:13:20.370 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-25T13:13:24.589 INFO:teuthology.orchestra.run.smithi069.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-25T13:13:24.822 INFO:teuthology.orchestra.run.smithi069.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.7 MB/s | 890 kB 00:00 2024-04-25T13:13:24.914 INFO:teuthology.orchestra.run.smithi069.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.5 MB/s | 145 kB 00:00 2024-04-25T13:13:25.023 INFO:teuthology.orchestra.run.smithi069.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 8.8 MB/s | 5.4 MB 00:00 2024-04-25T13:13:25.248 INFO:teuthology.orchestra.run.smithi069.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.6 MB/s | 1.5 MB 00:00 2024-04-25T13:13:25.307 INFO:teuthology.orchestra.run.smithi069.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-25T13:13:25.674 INFO:teuthology.orchestra.run.smithi069.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 4.7 MB 00:00 2024-04-25T13:13:26.018 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T13:13:26.741 INFO:teuthology.orchestra.run.smithi069.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 9.9 MB/s | 11 MB 00:01 2024-04-25T13:13:26.841 INFO:teuthology.orchestra.run.smithi069.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 252 kB/s | 25 kB 00:00 2024-04-25T13:13:27.169 INFO:teuthology.orchestra.run.smithi044.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T13:13:27.434 INFO:teuthology.orchestra.run.smithi069.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 7.0 MB/s | 21 MB 00:03 2024-04-25T13:13:27.958 INFO:teuthology.orchestra.run.smithi069.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 429 kB/s | 225 kB 00:00 2024-04-25T13:13:28.050 INFO:teuthology.orchestra.run.smithi069.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 295 kB/s | 27 kB 00:00 2024-04-25T13:13:28.142 INFO:teuthology.orchestra.run.smithi069.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 347 kB/s | 32 kB 00:00 2024-04-25T13:13:28.202 INFO:teuthology.orchestra.run.smithi044.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 861 kB/s | 890 kB 00:01 2024-04-25T13:13:28.210 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T13:13:28.285 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 393 kB/s | 890 kB 00:02 2024-04-25T13:13:28.385 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-immutable-object-cache-19.0.0-315 827 kB/s | 145 kB 00:00 2024-04-25T13:13:28.403 INFO:teuthology.orchestra.run.smithi044.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 725 kB/s | 145 kB 00:00 2024-04-25T13:13:28.542 INFO:teuthology.orchestra.run.smithi069.stdout:(14/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 5.2 MB/s | 17 MB 00:03 2024-04-25T13:13:28.575 INFO:teuthology.orchestra.run.smithi069.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.7 MB/s | 747 kB 00:00 2024-04-25T13:13:28.617 INFO:teuthology.orchestra.run.smithi069.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-25T13:13:28.643 INFO:teuthology.orchestra.run.smithi069.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-25T13:13:28.751 INFO:teuthology.orchestra.run.smithi069.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 3.7 MB/s | 500 kB 00:00 2024-04-25T13:13:28.786 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 6.6 MB/s | 21 MB 00:03 2024-04-25T13:13:28.827 INFO:teuthology.orchestra.run.smithi069.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 602 kB/s | 45 kB 00:00 2024-04-25T13:13:28.927 INFO:teuthology.orchestra.run.smithi069.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T13:13:29.020 INFO:teuthology.orchestra.run.smithi069.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T13:13:29.069 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 1.5 MB 00:00 2024-04-25T13:13:29.111 INFO:teuthology.orchestra.run.smithi069.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-25T13:13:29.345 INFO:teuthology.orchestra.run.smithi044.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T13:13:29.578 INFO:teuthology.orchestra.run.smithi044.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 2.4 MB 00:01 2024-04-25T13:13:29.678 INFO:teuthology.orchestra.run.smithi069.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 525 kB/s | 298 kB 00:00 2024-04-25T13:13:29.762 INFO:teuthology.orchestra.run.smithi069.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T13:13:29.787 INFO:teuthology.orchestra.run.smithi044.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 1.5 MB 00:00 2024-04-25T13:13:29.803 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-25T13:13:29.822 INFO:teuthology.orchestra.run.smithi069.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T13:13:30.862 INFO:teuthology.orchestra.run.smithi069.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 3.1 MB 00:01 2024-04-25T13:13:30.962 INFO:teuthology.orchestra.run.smithi069.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T13:13:31.054 INFO:teuthology.orchestra.run.smithi069.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-25T13:13:31.188 INFO:teuthology.orchestra.run.smithi069.stdout:(29/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.1 MB/s | 5.4 MB 00:02 2024-04-25T13:13:31.213 INFO:teuthology.orchestra.run.smithi069.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 896 kB/s | 142 kB 00:00 2024-04-25T13:13:31.637 INFO:teuthology.orchestra.run.smithi044.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 4.3 MB/s | 21 MB 00:04 2024-04-25T13:13:31.844 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 679 kB/s | 2.4 MB 00:03 2024-04-25T13:13:31.880 INFO:teuthology.orchestra.run.smithi044.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-25T13:13:31.988 INFO:teuthology.orchestra.run.smithi044.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 232 kB/s | 25 kB 00:00 2024-04-25T13:13:32.011 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 151 kB/s | 25 kB 00:00 2024-04-25T13:13:32.212 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.4 MB/s | 11 MB 00:02 2024-04-25T13:13:32.320 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 225 kB 00:00 2024-04-25T13:13:32.421 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T13:13:32.537 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 272 kB/s | 32 kB 00:00 2024-04-25T13:13:32.754 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.4 MB/s | 747 kB 00:00 2024-04-25T13:13:32.863 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T13:13:32.905 INFO:teuthology.orchestra.run.smithi069.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.1 MB/s | 3.5 MB 00:01 2024-04-25T13:13:32.955 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.4 MB/s | 126 kB 00:00 2024-04-25T13:13:33.147 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.5 MB/s | 500 kB 00:00 2024-04-25T13:13:33.155 INFO:teuthology.orchestra.run.smithi069.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-04-25T13:13:33.247 INFO:teuthology.orchestra.run.smithi069.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 539 kB/s | 49 kB 00:00 2024-04-25T13:13:33.348 INFO:teuthology.orchestra.run.smithi069.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 164 kB/s | 16 kB 00:00 2024-04-25T13:13:33.406 INFO:teuthology.orchestra.run.smithi069.stdout:(35/152): ceph-mgr-modules-core-19.0.0-3156.gd6 497 kB/s | 249 kB 00:00 2024-04-25T13:13:33.456 INFO:teuthology.orchestra.run.smithi069.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 273 kB 00:00 2024-04-25T13:13:33.548 INFO:teuthology.orchestra.run.smithi069.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 199 kB/s | 18 kB 00:00 2024-04-25T13:13:33.649 INFO:teuthology.orchestra.run.smithi069.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 805 kB/s | 80 kB 00:00 2024-04-25T13:13:33.732 INFO:teuthology.orchestra.run.smithi069.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T13:13:33.766 INFO:teuthology.orchestra.run.smithi069.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 195 kB 00:00 2024-04-25T13:13:33.808 INFO:teuthology.orchestra.run.smithi069.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T13:13:33.866 INFO:teuthology.orchestra.run.smithi069.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-25T13:13:33.900 INFO:teuthology.orchestra.run.smithi069.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T13:13:33.934 INFO:teuthology.orchestra.run.smithi069.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T13:13:33.976 INFO:teuthology.orchestra.run.smithi069.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T13:13:34.018 INFO:teuthology.orchestra.run.smithi069.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T13:13:34.068 INFO:teuthology.orchestra.run.smithi069.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T13:13:34.193 INFO:teuthology.orchestra.run.smithi069.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-04-25T13:13:34.244 INFO:teuthology.orchestra.run.smithi069.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T13:13:34.277 INFO:teuthology.orchestra.run.smithi069.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 647 kB/s | 21 kB 00:00 2024-04-25T13:13:34.311 INFO:teuthology.orchestra.run.smithi069.stdout:(51/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 843 kB/s | 762 kB 00:00 2024-04-25T13:13:34.336 INFO:teuthology.orchestra.run.smithi069.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T13:13:34.361 INFO:teuthology.orchestra.run.smithi069.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T13:13:34.428 INFO:teuthology.orchestra.run.smithi069.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-25T13:13:34.453 INFO:teuthology.orchestra.run.smithi069.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 205 kB/s | 18 kB 00:00 2024-04-25T13:13:34.470 INFO:teuthology.orchestra.run.smithi069.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-04-25T13:13:34.496 INFO:teuthology.orchestra.run.smithi069.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T13:13:34.513 INFO:teuthology.orchestra.run.smithi069.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T13:13:34.546 INFO:teuthology.orchestra.run.smithi069.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:13:34.580 INFO:teuthology.orchestra.run.smithi069.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.7 MB/s | 582 kB 00:00 2024-04-25T13:13:34.596 INFO:teuthology.orchestra.run.smithi069.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 518 kB/s | 25 kB 00:00 2024-04-25T13:13:34.622 INFO:teuthology.orchestra.run.smithi069.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T13:13:34.622 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 5.4 MB 00:01 2024-04-25T13:13:34.648 INFO:teuthology.orchestra.run.smithi069.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T13:13:34.673 INFO:teuthology.orchestra.run.smithi069.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T13:13:34.698 INFO:teuthology.orchestra.run.smithi069.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-25T13:13:34.731 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 417 kB/s | 45 kB 00:00 2024-04-25T13:13:34.790 INFO:teuthology.orchestra.run.smithi069.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-25T13:13:34.831 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T13:13:34.865 INFO:teuthology.orchestra.run.smithi069.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-25T13:13:34.873 INFO:teuthology.orchestra.run.smithi044.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.3 MB/s | 11 MB 00:03 2024-04-25T13:13:34.907 INFO:teuthology.orchestra.run.smithi069.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-25T13:13:34.949 INFO:teuthology.orchestra.run.smithi069.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T13:13:34.974 INFO:teuthology.orchestra.run.smithi069.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T13:13:35.007 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 922 kB/s | 161 kB 00:00 2024-04-25T13:13:35.063 INFO:teuthology.orchestra.run.smithi044.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 225 kB 00:00 2024-04-25T13:13:35.091 INFO:teuthology.orchestra.run.smithi069.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 41 MB/s | 5.8 MB 00:00 2024-04-25T13:13:35.133 INFO:teuthology.orchestra.run.smithi069.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-25T13:13:35.158 INFO:teuthology.orchestra.run.smithi069.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T13:13:35.239 INFO:teuthology.orchestra.run.smithi044.stdout:(12/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 17 MB 00:05 2024-04-25T13:13:35.264 INFO:teuthology.orchestra.run.smithi044.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 135 kB/s | 27 kB 00:00 2024-04-25T13:13:35.265 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 320 kB 00:00 2024-04-25T13:13:35.292 INFO:teuthology.orchestra.run.smithi069.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-25T13:13:35.318 INFO:teuthology.orchestra.run.smithi069.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T13:13:35.322 INFO:teuthology.orchestra.run.smithi044.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T13:13:35.343 INFO:teuthology.orchestra.run.smithi069.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T13:13:35.368 INFO:teuthology.orchestra.run.smithi069.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-04-25T13:13:35.393 INFO:teuthology.orchestra.run.smithi069.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T13:13:35.419 INFO:teuthology.orchestra.run.smithi069.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T13:13:35.431 INFO:teuthology.orchestra.run.smithi044.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T13:13:35.444 INFO:teuthology.orchestra.run.smithi069.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T13:13:35.474 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.4 MB/s | 298 kB 00:00 2024-04-25T13:13:35.532 INFO:teuthology.orchestra.run.smithi044.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T13:13:35.561 INFO:teuthology.orchestra.run.smithi069.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-25T13:13:35.586 INFO:teuthology.orchestra.run.smithi069.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-25T13:13:35.591 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 849 kB/s | 99 kB 00:00 2024-04-25T13:13:35.611 INFO:teuthology.orchestra.run.smithi069.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T13:13:35.637 INFO:teuthology.orchestra.run.smithi069.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T13:13:35.640 INFO:teuthology.orchestra.run.smithi044.stdout:(17/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 747 kB 00:00 2024-04-25T13:13:35.662 INFO:teuthology.orchestra.run.smithi069.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-25T13:13:35.687 INFO:teuthology.orchestra.run.smithi069.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T13:13:35.691 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 853 kB/s | 85 kB 00:00 2024-04-25T13:13:35.712 INFO:teuthology.orchestra.run.smithi069.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T13:13:35.738 INFO:teuthology.orchestra.run.smithi069.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T13:13:35.749 INFO:teuthology.orchestra.run.smithi044.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 2.3 MB/s | 500 kB 00:00 2024-04-25T13:13:35.754 INFO:teuthology.orchestra.run.smithi069.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T13:13:35.782 INFO:teuthology.orchestra.run.smithi069.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-25T13:13:35.841 INFO:teuthology.orchestra.run.smithi044.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 491 kB/s | 45 kB 00:00 2024-04-25T13:13:35.933 INFO:teuthology.orchestra.run.smithi044.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T13:13:35.955 INFO:teuthology.orchestra.run.smithi069.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 240 kB/s | 42 kB 00:00 2024-04-25T13:13:36.025 INFO:teuthology.orchestra.run.smithi044.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T13:13:36.147 INFO:teuthology.orchestra.run.smithi069.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-04-25T13:13:36.173 INFO:teuthology.orchestra.run.smithi069.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 205 kB/s | 44 kB 00:00 2024-04-25T13:13:36.198 INFO:teuthology.orchestra.run.smithi069.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T13:13:36.200 INFO:teuthology.orchestra.run.smithi044.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.8 MB/s | 320 kB 00:00 2024-04-25T13:13:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-04-25T13:13:36.232 INFO:teuthology.orchestra.run.smithi069.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T13:13:36.266 INFO:teuthology.orchestra.run.smithi069.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T13:13:36.283 INFO:teuthology.orchestra.run.smithi069.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T13:13:36.308 INFO:teuthology.orchestra.run.smithi069.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T13:13:36.325 INFO:teuthology.orchestra.run.smithi069.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T13:13:36.341 INFO:teuthology.orchestra.run.smithi069.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T13:13:36.359 INFO:teuthology.orchestra.run.smithi069.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 664 kB/s | 22 kB 00:00 2024-04-25T13:13:36.375 INFO:teuthology.orchestra.run.smithi044.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 298 kB 00:00 2024-04-25T13:13:36.417 INFO:teuthology.orchestra.run.smithi069.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T13:13:36.459 INFO:teuthology.orchestra.run.smithi044.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T13:13:36.543 INFO:teuthology.orchestra.run.smithi044.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T13:13:37.017 INFO:teuthology.orchestra.run.smithi069.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 515 kB/s | 309 kB 00:00 2024-04-25T13:13:37.051 INFO:teuthology.orchestra.run.smithi069.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 792 kB/s | 548 kB 00:00 2024-04-25T13:13:37.110 INFO:teuthology.orchestra.run.smithi069.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-25T13:13:37.201 INFO:teuthology.orchestra.run.smithi069.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-04-25T13:13:37.258 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.0 MB/s | 3.1 MB 00:01 2024-04-25T13:13:37.335 INFO:teuthology.orchestra.run.smithi069.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-04-25T13:13:37.425 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.0 MB/s | 170 kB 00:00 2024-04-25T13:13:37.485 INFO:teuthology.orchestra.run.smithi069.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T13:13:37.510 INFO:teuthology.orchestra.run.smithi069.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 385 kB/s | 67 kB 00:00 2024-04-25T13:13:37.517 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 374 kB/s | 34 kB 00:00 2024-04-25T13:13:37.586 INFO:teuthology.orchestra.run.smithi069.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T13:13:37.626 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.3 MB/s | 142 kB 00:00 2024-04-25T13:13:37.700 INFO:teuthology.orchestra.run.smithi069.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-25T13:13:37.753 INFO:teuthology.orchestra.run.smithi069.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-25T13:13:38.053 INFO:teuthology.orchestra.run.smithi044.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.0 MB/s | 3.1 MB 00:01 2024-04-25T13:13:38.127 INFO:teuthology.orchestra.run.smithi044.stdout:(27/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 5.4 MB 00:02 2024-04-25T13:13:38.187 INFO:teuthology.orchestra.run.smithi069.stdout:(114/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.3 MB/s | 49 MB 00:11 2024-04-25T13:13:38.212 INFO:teuthology.orchestra.run.smithi069.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 84 kB/s | 43 kB 00:00 2024-04-25T13:13:38.218 INFO:teuthology.orchestra.run.smithi044.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.0 MB/s | 170 kB 00:00 2024-04-25T13:13:38.237 INFO:teuthology.orchestra.run.smithi069.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 66 kB/s | 32 kB 00:00 2024-04-25T13:13:38.244 INFO:teuthology.orchestra.run.smithi044.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 293 kB/s | 34 kB 00:00 2024-04-25T13:13:38.311 INFO:teuthology.orchestra.run.smithi044.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T13:13:38.337 INFO:teuthology.orchestra.run.smithi069.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T13:13:38.362 INFO:teuthology.orchestra.run.smithi069.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-25T13:13:38.439 INFO:teuthology.orchestra.run.smithi069.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T13:13:38.463 INFO:teuthology.orchestra.run.smithi069.stdout:(120/152): python3-certifi-2021.10.8-2.el9.noar 54 kB/s | 15 kB 00:00 2024-04-25T13:13:38.488 INFO:teuthology.orchestra.run.smithi069.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 281 kB/s | 35 kB 00:00 2024-04-25T13:13:38.547 INFO:teuthology.orchestra.run.smithi069.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-25T13:13:38.589 INFO:teuthology.orchestra.run.smithi069.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T13:13:38.642 INFO:teuthology.orchestra.run.smithi069.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 114 kB/s | 11 kB 00:00 2024-04-25T13:13:38.675 INFO:teuthology.orchestra.run.smithi069.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 206 kB/s | 18 kB 00:00 2024-04-25T13:13:38.733 INFO:teuthology.orchestra.run.smithi069.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-04-25T13:13:38.767 INFO:teuthology.orchestra.run.smithi069.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-25T13:13:38.817 INFO:teuthology.orchestra.run.smithi069.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-25T13:13:38.917 INFO:teuthology.orchestra.run.smithi069.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T13:13:38.951 INFO:teuthology.orchestra.run.smithi069.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-25T13:13:39.009 INFO:teuthology.orchestra.run.smithi069.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 860 kB/s | 79 kB 00:00 2024-04-25T13:13:39.076 INFO:teuthology.orchestra.run.smithi069.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-04-25T13:13:39.118 INFO:teuthology.orchestra.run.smithi069.stdout:(133/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-04-25T13:13:39.152 INFO:teuthology.orchestra.run.smithi069.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-25T13:13:39.177 INFO:teuthology.orchestra.run.smithi069.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T13:13:39.261 INFO:teuthology.orchestra.run.smithi069.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 633 kB/s | 90 kB 00:00 2024-04-25T13:13:39.286 INFO:teuthology.orchestra.run.smithi069.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 211 kB/s | 28 kB 00:00 2024-04-25T13:13:39.311 INFO:teuthology.orchestra.run.smithi069.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-04-25T13:13:39.362 INFO:teuthology.orchestra.run.smithi069.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T13:13:39.412 INFO:teuthology.orchestra.run.smithi069.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-25T13:13:39.437 INFO:teuthology.orchestra.run.smithi069.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 285 kB/s | 36 kB 00:00 2024-04-25T13:13:39.462 INFO:teuthology.orchestra.run.smithi069.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-25T13:13:39.504 INFO:teuthology.orchestra.run.smithi069.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T13:13:39.543 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 17 MB 00:10 2024-04-25T13:13:39.555 INFO:teuthology.orchestra.run.smithi069.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 766 kB/s | 90 kB 00:00 2024-04-25T13:13:39.580 INFO:teuthology.orchestra.run.smithi069.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T13:13:39.601 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 1.8 MB/s | 3.5 MB 00:01 2024-04-25T13:13:39.605 INFO:teuthology.orchestra.run.smithi069.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T13:13:39.664 INFO:teuthology.orchestra.run.smithi069.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T13:13:39.739 INFO:teuthology.orchestra.run.smithi069.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T13:13:39.793 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.3 MB/s | 249 kB 00:00 2024-04-25T13:13:39.814 INFO:teuthology.orchestra.run.smithi069.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T13:13:39.910 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 423 kB/s | 49 kB 00:00 2024-04-25T13:13:39.914 INFO:teuthology.orchestra.run.smithi069.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T13:13:39.971 INFO:teuthology.orchestra.run.smithi044.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.0 MB/s | 3.5 MB 00:01 2024-04-25T13:13:40.019 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T13:13:40.263 INFO:teuthology.orchestra.run.smithi044.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 853 kB/s | 249 kB 00:00 2024-04-25T13:13:40.303 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 964 kB/s | 273 kB 00:00 2024-04-25T13:13:40.372 INFO:teuthology.orchestra.run.smithi044.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 456 kB/s | 49 kB 00:00 2024-04-25T13:13:40.481 INFO:teuthology.orchestra.run.smithi044.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T13:13:40.789 INFO:teuthology.orchestra.run.smithi044.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 885 kB/s | 273 kB 00:00 2024-04-25T13:13:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.3 MB/s | 762 kB 00:00 2024-04-25T13:13:40.994 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 169 kB/s | 18 kB 00:00 2024-04-25T13:13:41.103 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T13:13:41.178 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2024-04-25T13:13:41.220 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T13:13:41.262 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T13:13:41.329 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T13:13:41.369 INFO:teuthology.orchestra.run.smithi069.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 3.4 MB 00:01 2024-04-25T13:13:41.479 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 710 kB/s | 106 kB 00:00 2024-04-25T13:13:41.515 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.7 MB/s | 135 kB 00:00 2024-04-25T13:13:41.522 INFO:teuthology.orchestra.run.smithi044.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.0 MB/s | 762 kB 00:00 2024-04-25T13:13:41.548 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T13:13:41.590 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T13:13:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 13 MB/s | 689 kB 00:00 2024-04-25T13:13:41.673 INFO:teuthology.orchestra.run.smithi044.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-04-25T13:13:41.698 INFO:teuthology.orchestra.run.smithi044.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 104 kB/s | 18 kB 00:00 2024-04-25T13:13:41.699 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T13:13:41.733 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 107 kB 00:00 2024-04-25T13:13:41.759 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 858 kB/s | 21 kB 00:00 2024-04-25T13:13:41.800 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T13:13:41.826 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 33 kB 00:00 2024-04-25T13:13:41.832 INFO:teuthology.orchestra.run.smithi044.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-04-25T13:13:41.882 INFO:teuthology.orchestra.run.smithi044.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-25T13:13:41.907 INFO:teuthology.orchestra.run.smithi044.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-04-25T13:13:41.926 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T13:13:41.932 INFO:teuthology.orchestra.run.smithi044.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T13:13:41.959 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 551 kB/s | 18 kB 00:00 2024-04-25T13:13:41.983 INFO:teuthology.orchestra.run.smithi044.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T13:13:41.993 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-04-25T13:13:42.008 INFO:teuthology.orchestra.run.smithi044.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T13:13:42.027 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-25T13:13:42.033 INFO:teuthology.orchestra.run.smithi044.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T13:13:42.059 INFO:teuthology.orchestra.run.smithi044.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T13:13:42.060 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T13:13:42.084 INFO:teuthology.orchestra.run.smithi044.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T13:13:42.109 INFO:teuthology.orchestra.run.smithi044.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T13:13:42.111 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T13:13:42.151 INFO:teuthology.orchestra.run.smithi044.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T13:13:42.153 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-04-25T13:13:42.176 INFO:teuthology.orchestra.run.smithi044.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T13:13:42.186 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 770 kB/s | 25 kB 00:00 2024-04-25T13:13:42.202 INFO:teuthology.orchestra.run.smithi044.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-04-25T13:13:42.227 INFO:teuthology.orchestra.run.smithi044.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T13:13:42.252 INFO:teuthology.orchestra.run.smithi044.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T13:13:42.303 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-04-25T13:13:42.336 INFO:teuthology.orchestra.run.smithi044.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T13:13:42.345 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T13:13:42.361 INFO:teuthology.orchestra.run.smithi044.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 172 kB/s | 18 kB 00:00 2024-04-25T13:13:42.387 INFO:teuthology.orchestra.run.smithi044.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T13:13:42.387 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T13:13:42.412 INFO:teuthology.orchestra.run.smithi044.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T13:13:42.438 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-25T13:13:42.438 INFO:teuthology.orchestra.run.smithi044.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T13:13:42.496 INFO:teuthology.orchestra.run.smithi044.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-25T13:13:42.605 INFO:teuthology.orchestra.run.smithi044.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-25T13:13:42.622 INFO:teuthology.orchestra.run.smithi044.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-04-25T13:13:42.663 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T13:13:42.672 INFO:teuthology.orchestra.run.smithi044.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 954 kB/s | 47 kB 00:00 2024-04-25T13:13:42.681 INFO:teuthology.orchestra.run.smithi069.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 3.1 MB 00:03 2024-04-25T13:13:42.684 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:13:42.685 INFO:teuthology.orchestra.run.smithi069.stdout:Total 8.9 MB/s | 222 MB 00:24 2024-04-25T13:13:42.697 INFO:teuthology.orchestra.run.smithi044.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T13:13:42.731 INFO:teuthology.orchestra.run.smithi044.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T13:13:42.764 INFO:teuthology.orchestra.run.smithi044.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-25T13:13:42.863 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T13:13:42.923 INFO:teuthology.orchestra.run.smithi044.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T13:13:42.947 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-25T13:13:43.005 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-25T13:13:43.039 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.9 MB/s | 133 kB 00:00 2024-04-25T13:13:43.049 INFO:teuthology.orchestra.run.smithi044.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T13:13:43.091 INFO:teuthology.orchestra.run.smithi044.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.7 MB/s | 1.6 MB 00:00 2024-04-25T13:13:43.133 INFO:teuthology.orchestra.run.smithi044.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T13:13:43.158 INFO:teuthology.orchestra.run.smithi044.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T13:13:43.189 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-04-25T13:13:43.257 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T13:13:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.5 MB/s | 87 kB 00:00 2024-04-25T13:13:43.300 INFO:teuthology.orchestra.run.smithi044.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-04-25T13:13:43.333 INFO:teuthology.orchestra.run.smithi044.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-04-25T13:13:43.359 INFO:teuthology.orchestra.run.smithi044.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T13:13:43.540 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-04-25T13:13:43.551 INFO:teuthology.orchestra.run.smithi044.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-04-25T13:13:43.576 INFO:teuthology.orchestra.run.smithi044.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T13:13:43.582 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-25T13:13:43.601 INFO:teuthology.orchestra.run.smithi044.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T13:13:43.624 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-25T13:13:43.627 INFO:teuthology.orchestra.run.smithi044.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-25T13:13:43.652 INFO:teuthology.orchestra.run.smithi044.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T13:13:43.658 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 772 kB/s | 26 kB 00:00 2024-04-25T13:13:43.685 INFO:teuthology.orchestra.run.smithi044.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T13:13:43.691 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 577 kB/s | 19 kB 00:00 2024-04-25T13:13:43.711 INFO:teuthology.orchestra.run.smithi044.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 600 kB/s | 35 kB 00:00 2024-04-25T13:13:43.734 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T13:13:43.790 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-04-25T13:13:43.801 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 523 kB/s | 35 kB 00:00 2024-04-25T13:13:43.930 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-04-25T13:13:43.930 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-04-25T13:13:43.936 INFO:teuthology.orchestra.run.smithi044.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-25T13:13:43.961 INFO:teuthology.orchestra.run.smithi044.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-04-25T13:13:43.995 INFO:teuthology.orchestra.run.smithi044.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-25T13:13:44.018 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-25T13:13:44.020 INFO:teuthology.orchestra.run.smithi044.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T13:13:44.045 INFO:teuthology.orchestra.run.smithi044.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 847 kB/s | 42 kB 00:00 2024-04-25T13:13:44.068 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T13:13:44.071 INFO:teuthology.orchestra.run.smithi044.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T13:13:44.105 INFO:teuthology.orchestra.run.smithi044.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T13:13:44.118 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-25T13:13:44.130 INFO:teuthology.orchestra.run.smithi044.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-25T13:13:44.164 INFO:teuthology.orchestra.run.smithi044.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 874 kB/s | 51 kB 00:00 2024-04-25T13:13:44.177 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T13:13:44.189 INFO:teuthology.orchestra.run.smithi044.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-04-25T13:13:44.252 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-25T13:13:44.327 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T13:13:44.386 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T13:13:44.464 INFO:teuthology.orchestra.run.smithi044.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 151 kB/s | 42 kB 00:00 2024-04-25T13:13:44.478 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:04 2024-04-25T13:13:44.503 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T13:13:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T13:13:44.554 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T13:13:44.821 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 157 kB/s | 42 kB 00:00 2024-04-25T13:13:44.931 INFO:teuthology.orchestra.run.smithi044.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T13:13:44.948 INFO:teuthology.orchestra.run.smithi044.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 92 kB/s | 44 kB 00:00 2024-04-25T13:13:44.990 INFO:teuthology.orchestra.run.smithi044.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T13:13:45.007 INFO:teuthology.orchestra.run.smithi044.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-04-25T13:13:45.038 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 206 kB/s | 44 kB 00:00 2024-04-25T13:13:45.082 INFO:teuthology.orchestra.run.smithi044.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-04-25T13:13:45.116 INFO:teuthology.orchestra.run.smithi044.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-25T13:13:45.133 INFO:teuthology.orchestra.run.smithi044.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T13:13:45.174 INFO:teuthology.orchestra.run.smithi044.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T13:13:45.200 INFO:teuthology.orchestra.run.smithi044.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-04-25T13:13:45.225 INFO:teuthology.orchestra.run.smithi044.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 877 kB/s | 45 kB 00:00 2024-04-25T13:13:45.251 INFO:teuthology.orchestra.run.smithi044.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T13:13:45.288 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T13:13:45.293 INFO:teuthology.orchestra.run.smithi044.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T13:13:45.313 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 805 kB/s | 221 kB 00:00 2024-04-25T13:13:45.338 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 727 kB/s | 36 kB 00:00 2024-04-25T13:13:45.364 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-25T13:13:45.389 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T13:13:45.431 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 871 kB/s | 58 kB 00:00 2024-04-25T13:13:45.456 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T13:13:45.490 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T13:13:45.515 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-04-25T13:13:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-25T13:13:45.691 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-25T13:13:45.744 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-04-25T13:13:45.745 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-04-25T13:13:45.809 INFO:teuthology.orchestra.run.smithi044.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-04-25T13:13:45.843 INFO:teuthology.orchestra.run.smithi044.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 928 kB/s | 548 kB 00:00 2024-04-25T13:13:45.849 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-04-25T13:13:45.892 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-04-25T13:13:45.917 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-25T13:13:45.984 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-25T13:13:46.077 INFO:teuthology.orchestra.run.smithi044.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-25T13:13:46.084 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-25T13:13:46.126 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T13:13:46.184 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T13:13:46.352 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T13:13:46.352 INFO:teuthology.orchestra.run.smithi044.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 94 kB/s | 26 kB 00:00 2024-04-25T13:13:46.518 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T13:13:46.560 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-25T13:13:46.602 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 771 kB/s | 32 kB 00:00 2024-04-25T13:13:46.644 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 362 kB/s | 15 kB 00:00 2024-04-25T13:13:46.694 INFO:teuthology.orchestra.run.smithi044.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T13:13:46.778 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T13:13:46.894 INFO:teuthology.orchestra.run.smithi044.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-04-25T13:13:46.895 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T13:13:46.987 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-04-25T13:13:47.012 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T13:13:47.037 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-04-25T13:13:47.061 INFO:teuthology.orchestra.run.smithi044.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 184 kB/s | 67 kB 00:00 2024-04-25T13:13:47.079 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-25T13:13:47.138 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T13:13:47.171 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 322 kB/s | 11 kB 00:00 2024-04-25T13:13:47.247 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-25T13:13:47.280 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 696 kB/s | 23 kB 00:00 2024-04-25T13:13:47.328 INFO:teuthology.orchestra.run.smithi044.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-04-25T13:13:47.397 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-25T13:13:47.656 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 76 kB/s | 20 kB 00:00 2024-04-25T13:13:47.706 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-04-25T13:13:47.745 INFO:teuthology.orchestra.run.smithi044.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T13:13:47.756 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 923 kB/s | 46 kB 00:00 2024-04-25T13:13:47.815 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T13:13:47.874 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 989 kB/s | 58 kB 00:00 2024-04-25T13:13:47.937 INFO:teuthology.orchestra.run.smithi044.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 901 kB/s | 548 kB 00:00 2024-04-25T13:13:47.966 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-04-25T13:13:47.991 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T13:13:48.016 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-25T13:13:48.050 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-25T13:13:48.075 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-25T13:13:48.087 INFO:teuthology.orchestra.run.smithi044.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-25T13:13:48.100 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-04-25T13:13:48.142 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-25T13:13:48.167 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-04-25T13:13:48.209 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 866 kB/s | 36 kB 00:00 2024-04-25T13:13:48.235 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 767 kB/s | 51 kB 00:00 2024-04-25T13:13:48.347 INFO:teuthology.orchestra.run.smithi044.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 78 kB/s | 32 kB 00:00 2024-04-25T13:13:48.360 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2024-04-25T13:13:48.371 INFO:teuthology.orchestra.run.smithi044.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 53 kB/s | 15 kB 00:00 2024-04-25T13:13:48.385 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-25T13:13:48.427 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-04-25T13:13:48.494 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-25T13:13:48.545 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T13:13:48.836 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-25T13:13:48.855 INFO:teuthology.orchestra.run.smithi044.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 339 kB/s | 172 kB 00:00 2024-04-25T13:13:48.880 INFO:teuthology.orchestra.run.smithi044.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-25T13:13:49.147 INFO:teuthology.orchestra.run.smithi044.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-25T13:13:49.373 INFO:teuthology.orchestra.run.smithi044.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 586 kB/s | 303 kB 00:00 2024-04-25T13:13:49.656 INFO:teuthology.orchestra.run.smithi044.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-25T13:13:50.081 INFO:teuthology.orchestra.run.smithi044.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-25T13:13:50.131 INFO:teuthology.orchestra.run.smithi044.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.7 MB/s | 2.0 MB 00:00 2024-04-25T13:13:50.274 INFO:teuthology.orchestra.run.smithi044.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T13:13:50.399 INFO:teuthology.orchestra.run.smithi044.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-25T13:13:50.541 INFO:teuthology.orchestra.run.smithi044.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-04-25T13:13:50.561 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.7 MB/s | 3.4 MB 00:02 2024-04-25T13:13:50.666 INFO:teuthology.orchestra.run.smithi044.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T13:13:50.770 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-25T13:13:50.816 INFO:teuthology.orchestra.run.smithi044.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 71 kB/s | 20 kB 00:00 2024-04-25T13:13:50.879 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T13:13:51.166 INFO:teuthology.orchestra.run.smithi044.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-25T13:13:51.367 INFO:teuthology.orchestra.run.smithi044.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T13:13:51.517 INFO:teuthology.orchestra.run.smithi044.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-25T13:13:51.717 INFO:teuthology.orchestra.run.smithi044.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-04-25T13:13:51.883 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-04-25T13:13:51.968 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:13:51.984 INFO:teuthology.orchestra.run.smithi044.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-04-25T13:13:52.006 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:13:52.026 INFO:teuthology.orchestra.run.smithi044.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-25T13:13:52.060 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:13:52.094 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:13:52.119 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:13:52.162 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:13:52.285 INFO:teuthology.orchestra.run.smithi044.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-04-25T13:13:52.410 INFO:teuthology.orchestra.run.smithi044.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-25T13:13:52.480 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:13:52.543 INFO:teuthology.orchestra.run.smithi044.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-25T13:13:52.595 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:13:52.718 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:13:52.746 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:13:52.837 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:13:52.844 INFO:teuthology.orchestra.run.smithi044.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 435 kB/s | 188 kB 00:00 2024-04-25T13:13:52.867 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:13:52.877 INFO:teuthology.orchestra.run.smithi044.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 177 kB/s | 59 kB 00:00 2024-04-25T13:13:53.022 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:13:53.072 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:13:53.100 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:13:53.119 INFO:teuthology.orchestra.run.smithi044.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-25T13:13:53.203 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:13:53.219 INFO:teuthology.orchestra.run.smithi044.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-25T13:13:53.226 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:13:53.289 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:13:53.310 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:13:53.332 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:13:53.528 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:13:53.560 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:13:53.570 INFO:teuthology.orchestra.run.smithi044.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 256 kB/s | 90 kB 00:00 2024-04-25T13:13:53.637 INFO:teuthology.orchestra.run.smithi044.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 444 kB/s | 230 kB 00:00 2024-04-25T13:13:53.668 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:13:53.762 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:13:53.792 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:13:53.827 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:13:53.891 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:13:53.904 INFO:teuthology.orchestra.run.smithi044.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-25T13:13:53.926 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:13:53.969 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:13:54.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:13:54.030 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:13:54.098 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:13:54.152 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:13:54.154 INFO:teuthology.orchestra.run.smithi044.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 732 kB/s | 427 kB 00:00 2024-04-25T13:13:54.183 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:13:54.218 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:13:54.293 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:13:54.329 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:13:54.372 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:13:54.401 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:13:54.413 INFO:teuthology.orchestra.run.smithi044.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-25T13:13:54.470 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:13:54.516 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:13:54.589 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:13:54.730 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:13:54.809 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:13:54.913 INFO:teuthology.orchestra.run.smithi044.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T13:13:55.326 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:13:55.405 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:13:55.443 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:13:55.477 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:13:55.515 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:13:55.557 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:13:55.702 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:13:55.768 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:13:55.822 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:13:55.895 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:13:55.927 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:13:55.937 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 453 kB/s | 3.1 MB 00:07 2024-04-25T13:13:55.955 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:13:56.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:13:56.038 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:13:56.080 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:13:56.123 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:13:56.259 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:13:56.304 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:13:56.334 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:13:56.374 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:13:56.430 INFO:teuthology.orchestra.run.smithi044.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.7 MB/s | 3.4 MB 00:02 2024-04-25T13:13:56.469 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:13:56.509 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:13:56.513 INFO:teuthology.orchestra.run.smithi044.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T13:13:56.548 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:13:56.574 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:13:56.599 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:13:56.630 INFO:teuthology.orchestra.run.smithi044.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T13:13:56.673 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:13:56.793 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:13:56.823 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:13:56.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:13:56.877 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:13:56.916 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:13:56.948 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:13:56.977 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:13:57.012 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:13:57.045 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:13:57.087 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:13:57.132 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:13:57.216 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:13:57.260 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:13:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:13:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:13:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:13:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:13:57.956 INFO:teuthology.orchestra.run.smithi044.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 3.1 MB 00:03 2024-04-25T13:13:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:13:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:13:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:13:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:13:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:13:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:13:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:13:58.720 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:13:58.750 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:13:58.826 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:13:59.460 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:13:59.579 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:14:00.784 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:14:00.877 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:14:00.904 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:14:00.938 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:14:01.249 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:14:01.284 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:01.375 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:01.405 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:14:01.441 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:14:01.919 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:14:01.971 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:02.042 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:02.072 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:14:03.556 INFO:teuthology.orchestra.run.smithi044.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.6 MB/s | 49 MB 00:31 2024-04-25T13:14:03.561 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:14:03.561 INFO:teuthology.orchestra.run.smithi044.stdout:Total 5.1 MB/s | 222 MB 00:43 2024-04-25T13:14:04.300 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:04.385 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:04.694 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-25T13:14:04.852 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-25T13:14:04.853 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-25T13:14:04.923 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:04.971 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:05.062 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:05.063 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:14:05.063 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:14:05.063 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:05.098 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:06.729 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-25T13:14:06.729 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-25T13:14:08.945 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 1.3 MB/s | 49 MB 00:36 2024-04-25T13:14:08.949 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:14:08.949 INFO:teuthology.orchestra.run.smithi037.stdout:Total 4.4 MB/s | 222 MB 00:50 2024-04-25T13:14:10.034 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T13:14:10.194 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T13:14:10.194 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T13:14:12.082 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T13:14:12.082 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T13:14:12.902 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-25T13:14:13.001 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:14:13.040 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:14:13.102 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:14:13.136 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:14:13.160 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:14:13.204 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:14:13.549 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:14:13.662 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:14:13.786 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:14:13.821 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:14:13.930 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:14:13.959 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:14:14.116 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:14:14.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:14:14.200 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:14:14.297 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:14:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:14:14.406 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:14:14.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:14:14.460 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:14:14.671 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:14:14.702 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:14:14.821 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:14:14.918 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:14:14.951 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:14:14.986 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:14:15.062 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:14:15.102 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:14:15.146 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:14:15.195 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:14:15.231 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:14:15.303 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:14:15.362 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:14:15.392 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:14:15.427 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:14:15.506 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:14:15.538 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:14:15.582 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:14:15.618 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:14:15.691 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:14:15.742 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:14:15.819 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:14:15.966 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:14:16.020 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-04-25T13:14:16.496 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-04-25T13:14:16.497 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:16.577 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:14:16.667 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:14:16.668 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:16.683 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:16.683 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:16.683 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:14:16.684 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:16.701 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:14:16.745 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:14:16.782 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:14:16.825 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:14:17.015 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:14:17.077 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:14:17.134 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:14:17.208 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:14:17.244 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:14:17.281 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:14:17.329 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:14:17.364 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:14:17.416 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:14:17.475 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:14:17.619 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:14:17.663 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:14:17.693 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:14:17.735 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:14:17.834 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:14:17.868 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:14:17.908 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:14:17.950 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:14:17.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:14:18.053 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:14:18.154 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:18.179 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:18.192 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:14:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T13:14:18.232 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:14:18.260 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:14:18.295 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:14:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T13:14:18.334 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:14:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T13:14:18.374 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:14:18.403 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:14:18.413 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:18.436 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:18.438 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:14:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T13:14:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T13:14:18.479 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:14:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T13:14:18.522 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:14:18.555 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:14:18.647 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:14:18.703 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:14:18.734 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:14:18.782 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:14:18.863 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T13:14:18.872 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:18.895 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:14:18.953 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:14:18.967 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:14:18.994 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:14:19.050 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:14:19.091 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:14:19.114 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:14:19.126 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:14:19.218 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:14:19.256 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:14:19.300 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:14:19.348 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:14:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T13:14:19.426 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:14:19.479 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T13:14:19.506 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:14:19.514 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:14:19.611 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:14:19.640 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:14:19.651 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:14:19.686 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T13:14:19.689 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:14:19.707 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:14:19.720 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:14:19.721 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:14:19.731 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T13:14:19.749 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:14:19.766 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:14:19.800 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:14:19.903 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:14:19.916 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:14:19.949 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T13:14:19.968 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:14:20.058 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:14:20.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T13:14:20.092 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:20.093 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:14:20.122 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:20.158 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T13:14:20.163 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:14:20.190 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T13:14:20.224 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:14:20.234 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T13:14:20.263 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:20.278 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:14:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:14:20.310 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T13:14:20.349 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T13:14:20.387 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:14:20.393 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T13:14:20.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T13:14:20.453 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T13:14:20.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T13:14:20.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T13:14:20.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T13:14:20.650 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T13:14:20.728 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T13:14:20.760 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T13:14:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T13:14:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T13:14:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T13:14:20.956 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T13:14:21.034 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T13:14:21.073 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:14:21.190 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T13:14:21.194 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:14:21.243 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T13:14:21.292 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:21.330 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:21.806 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T13:14:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T13:14:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T13:14:21.984 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T13:14:22.022 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T13:14:22.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T13:14:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T13:14:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T13:14:22.286 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:22.315 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T13:14:22.406 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T13:14:22.415 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:22.442 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T13:14:22.478 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T13:14:22.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:14:22.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T13:14:22.561 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T13:14:22.568 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:14:22.595 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:14:22.605 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T13:14:22.608 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:22.622 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:14:22.631 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:22.631 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:22.631 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:14:22.632 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:22.632 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:22.632 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:22.656 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T13:14:22.664 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:14:22.675 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:22.687 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:22.783 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:14:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T13:14:22.840 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:14:22.844 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T13:14:22.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T13:14:22.916 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T13:14:23.001 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:14:23.015 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T13:14:23.043 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:23.048 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T13:14:23.088 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T13:14:23.114 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T13:14:23.134 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:23.138 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T13:14:23.165 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:14:23.202 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:14:23.218 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T13:14:23.347 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T13:14:23.396 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T13:14:23.424 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T13:14:23.459 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T13:14:23.498 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T13:14:23.538 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T13:14:23.567 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T13:14:23.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T13:14:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T13:14:23.677 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T13:14:23.706 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:14:23.711 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T13:14:23.738 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:23.758 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:14:23.794 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:23.795 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T13:14:23.809 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:23.817 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:23.839 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:14:23.842 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T13:14:23.862 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:23.873 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T13:14:23.884 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:23.884 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:23.885 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:14:23.885 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:23.921 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T13:14:23.929 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:23.951 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:23.951 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:23.952 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:14:23.952 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:23.952 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:23.952 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:24.033 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T13:14:24.093 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T13:14:24.254 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:24.277 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:24.277 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:24.277 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:14:24.277 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:24.278 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:24.278 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:24.869 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T13:14:24.913 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T13:14:24.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T13:14:25.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T13:14:25.106 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T13:14:25.196 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T13:14:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T13:14:25.408 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T13:14:25.446 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T13:14:25.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T13:14:26.189 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:26.215 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T13:14:26.219 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:26.341 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T13:14:26.789 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:26.831 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:26.924 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:26.924 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:14:26.925 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:14:26.925 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:26.957 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T13:14:27.734 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T13:14:27.762 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T13:14:27.795 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T13:14:28.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T13:14:28.166 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:28.249 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T13:14:28.279 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T13:14:28.316 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T13:14:28.820 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T13:14:28.833 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:14:28.852 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:28.874 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:14:28.915 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:14:28.923 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T13:14:28.946 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T13:14:28.947 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:14:28.974 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:14:29.158 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:14:29.195 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:14:29.225 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:14:29.805 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:29.840 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:29.860 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:30.711 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:14:30.785 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:14:30.827 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:14:30.910 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:14:30.940 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:14:30.940 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:31.032 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:31.065 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:14:31.065 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:31.161 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:31.230 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:14:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:31.385 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:31.956 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T13:14:31.997 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:32.093 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T13:14:32.093 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T13:14:32.093 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T13:14:32.093 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:32.123 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:14:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:14:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:14:36.868 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:14:36.869 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:14:36.870 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:14:36.871 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:14:36.872 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:14:36.873 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:14:36.874 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:14:36.875 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:14:36.876 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:14:36.877 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:14:36.878 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:14:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:14:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:14:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:14:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:14:36.879 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-04-25T13:14:38.972 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.973 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.974 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.975 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:14:38.976 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:14:38.977 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:14:38.978 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:14:38.979 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:14:38.980 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:14:38.981 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:14:38.982 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:14:38.983 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:14:38.984 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:14:38.984 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:14:38.984 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-04-25T13:14:39.064 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-25T13:14:39.065 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:39.245 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:39.262 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:39.262 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:14:39.262 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:39.354 DEBUG:teuthology.parallel:result is None 2024-04-25T13:14:40.790 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:40.817 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:40.817 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:40.818 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:14:40.818 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:40.818 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:40.818 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:41.049 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:41.072 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:41.073 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:41.073 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:14:41.073 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:41.073 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:41.073 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:41.533 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:41.557 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:41.557 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:41.557 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:14:41.557 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:41.558 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:41.558 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:41.664 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:14:41.722 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:14:41.785 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:14:41.996 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:14:42.043 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:14:42.124 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:14:42.213 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:14:42.363 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:14:42.401 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:14:42.433 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:14:42.460 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:14:42.914 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:42.942 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:42.975 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:14:43.039 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:14:43.074 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:43.976 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-25T13:14:43.977 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:44.160 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:44.171 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:44.179 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T13:14:44.180 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:44.180 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T13:14:44.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:44.217 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:45.234 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:45.279 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:45.384 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:45.555 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:14:45.593 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:45.611 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:45.611 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:45.612 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:14:45.612 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:45.612 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:45.612 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:45.663 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:45.674 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:45.705 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:45.732 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T13:14:45.732 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:45.732 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T13:14:45.733 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:45.733 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T13:14:45.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:45.776 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:14:45.829 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:14:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:45.987 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T13:14:45.987 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:45.988 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T13:14:45.988 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:45.988 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T13:14:45.988 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T13:14:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T13:14:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T13:14:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T13:14:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:14:46.831 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:46.855 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:46.855 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:46.855 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:14:46.856 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:46.856 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:46.856 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:46.907 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T13:14:46.930 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:46.930 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:46.930 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:14:46.930 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T13:14:46.983 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:47.006 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:47.006 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:47.007 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:14:47.007 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:47.007 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:47.007 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T13:14:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:14:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:14:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T13:14:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:47.349 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:47.349 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:47.349 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:14:47.349 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:47.349 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:47.350 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:14:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T13:14:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T13:14:47.821 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:47.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T13:14:47.864 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T13:14:47.928 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T13:14:47.955 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:49.052 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T13:14:49.090 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:50.108 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T13:14:50.132 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:50.242 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T13:14:50.411 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T13:14:50.442 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T13:14:50.467 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:50.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:50.529 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T13:14:50.631 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T13:14:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T13:14:51.659 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T13:14:51.687 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:51.710 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T13:14:51.710 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:51.710 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T13:14:51.711 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:51.711 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:14:51.711 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:51.755 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T13:14:51.777 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T13:14:51.778 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:51.822 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:51.845 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T13:14:51.846 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:51.846 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T13:14:51.846 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:51.846 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:14:51.846 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:52.155 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:52.179 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T13:14:52.180 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T13:14:52.180 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T13:14:52.180 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:52.180 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T13:14:52.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:14:52.204 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:14:52.246 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:14:52.287 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:14:52.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:14:52.345 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:14:52.540 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:14:52.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:14:52.613 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:14:53.253 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:53.294 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:53.315 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:54.231 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:14:54.324 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:14:54.367 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:14:54.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:14:54.504 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:54.596 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:14:54.629 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:54.732 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:54.811 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:14:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T13:14:57.076 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T13:14:57.118 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T13:14:57.149 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T13:14:57.176 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T13:14:57.378 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T13:14:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T13:14:57.452 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T13:14:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T13:14:59.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T13:14:59.154 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T13:14:59.197 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T13:14:59.291 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T13:14:59.325 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T13:14:59.325 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:59.401 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T13:14:59.434 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:14:59.434 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:59.519 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T13:14:59.594 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:15:00.695 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:15:00.696 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:15:00.697 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:15:00.698 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:15:00.699 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:15:00.700 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:15:00.701 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:15:00.702 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:15:00.703 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:15:00.704 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:15:00.705 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:15:00.706 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-25T13:15:02.921 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.922 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.923 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.924 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:15:02.925 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:15:02.926 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:15:02.927 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:15:02.928 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:15:02.929 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:15:02.930 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.931 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:15:02.932 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:15:02.933 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-25T13:15:02.933 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-25T13:15:03.359 DEBUG:teuthology.parallel:result is None 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T13:15:05.317 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T13:15:05.318 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T13:15:05.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T13:15:05.320 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T13:15:05.321 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T13:15:05.322 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T13:15:05.323 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T13:15:05.324 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T13:15:05.325 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T13:15:05.326 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T13:15:05.327 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T13:15:05.328 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T13:15:05.329 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T13:15:07.418 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T13:15:07.418 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:15:07.418 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-25T13:15:07.418 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.419 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T13:15:07.421 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T13:15:07.422 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T13:15:07.423 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T13:15:07.424 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T13:15:07.425 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T13:15:07.426 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.427 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T13:15:07.428 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T13:15:07.429 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:15:07.430 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T13:15:07.785 DEBUG:teuthology.parallel:result is None 2024-04-25T13:15:07.786 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:08.033 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:15:08.054 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.055 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.055 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:15:08.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:08.299 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:15:08.326 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.327 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.327 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:15:08.330 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:08.468 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T13:15:08.494 INFO:teuthology.orchestra.run.smithi069.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.494 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T13:15:08.494 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T13:15:08.496 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:15:08.496 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:15:08.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:15:08.524 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:15:08.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:15:08.556 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:15:08.556 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:15:08.587 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:15:08.587 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:15:08.587 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:15:08.612 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:15:08.680 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:15:08.680 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:15:08.709 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:15:08.777 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:15:08.777 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:15:08.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:15:08.874 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:15:08.874 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:15:08.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:15:08.899 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:15:08.966 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:15:08.966 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:15:08.995 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:15:09.062 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:15:09.062 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:15:09.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:15:09.149 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:15:09.149 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:15:09.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:15:09.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:15:09.239 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:15:09.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:15:09.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:15:09.337 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:15:09.337 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:15:09.361 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:15:09.424 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T13:15:09.531 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:15:09.532 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:09.532 INFO:tasks.cephadm:Cluster fsid is d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:15:09.532 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T13:15:09.532 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.44', 'mon.c': '172.21.15.69'} 2024-04-25T13:15:09.533 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-04-25T13:15:09.533 INFO:tasks.cephadm:First mgr is x 2024-04-25T13:15:09.533 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T13:15:09.533 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-25T13:15:09.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-04-25T13:15:09.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-04-25T13:15:09.611 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T13:15:09.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:09.750 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T13:15:09.884 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:15:09.885 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T13:15:09.886 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:10.475 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:15 /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:10.476 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.081 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:15 /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.081 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.520 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 13:15 /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.521 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.540 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.559 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T13:15:11.593 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T13:15:11.594 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:15:11.596 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:15:11.602 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T13:15:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:15:13.132 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:15:13.517 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:15:41.442 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T13:15:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:15:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:15:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-04-25T13:15:41.442 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:15:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-04-25T13:15:41.443 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T13:15:43.753 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-04-25T13:15:43.754 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-25T13:15:49.433 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:15:49.434 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:15:49.460 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-25T13:15:49.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-04-25T13:15:49.534 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-04-25T13:15:49.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-25T13:15:49.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-04-25T13:15:49.619 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-04-25T13:15:49.646 INFO:tasks.cephadm:Writing seed config... 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T13:15:49.647 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T13:15:49.648 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T13:15:49.649 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T13:15:49.650 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T13:15:49.651 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:15:49.651 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T13:15:49.667 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d766aeb0-0305-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T13:15:49.667 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T13:15:49.709 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T13:15:49.751 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T13:15:49.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:15:49.947 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T13:15:49.947 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'd766aeb0-0305-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-04-25T13:15:49.948 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T13:15:49.948 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-04-25T13:15:49.975 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:15:49.975 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-04-25T13:15:49.975 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-04-25T13:15:49.982 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:15:49.982 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:15:49.989 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:15:49.989 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:15:49.995 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-25T13:15:50.002 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-25T13:15:50.002 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:15:50.002 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-04-25T13:15:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T13:15:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T13:15:50.026 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-04-25T13:15:50.027 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-04-25T13:15:50.032 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T13:15:50.032 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T13:15:50.038 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T13:15:50.038 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:15:50.045 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-25T13:15:50.051 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-25T13:15:50.051 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-25T13:15:50.051 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-04-25T13:15:50.051 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:15:50.051 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140103888366752 on /run/cephadm/d766aeb0-0305-11ef-bc93-c7b262605968.lock 2024-04-25T13:15:50.052 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140103888366752 acquired on /run/cephadm/d766aeb0-0305-11ef-bc93-c7b262605968.lock 2024-04-25T13:15:50.052 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-04-25T13:15:50.052 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-04-25T13:15:50.052 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-25T13:15:50.055 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-04-25T13:15:50.055 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-04-25T13:15:50.058 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T13:15:50.058 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T13:15:50.058 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T13:15:50.060 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T13:15:50.061 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-25T13:15:50.062 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-25T13:15:50.062 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T13:15:50.062 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T13:15:50.063 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T13:15:50.570 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T13:15:51.382 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:15:51.382 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T13:15:51.383 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T13:15:52.363 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-04-25T13:15:52.364 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-04-25T13:15:53.310 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCIVypmYyOqKRAAri8z9TDyn4l+LiUqy7NNOA== 2024-04-25T13:15:54.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCJVypm/2fkKhAAZDoyA3rAdnnlR6BmVMCn7w== 2024-04-25T13:15:55.116 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQCKVypmV7KVJRAAAJ7ZK3RadLsyAalJqq22ag== 2024-04-25T13:15:55.116 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T13:15:57.859 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-25T13:15:57.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:15:57.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T13:15:57.860 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:15:57.860 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-04-25T13:16:06.773 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-04-25T13:16:07.282 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T13:16:07.552 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T13:16:07.965 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target. 2024-04-25T13:16:07.965 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target. 2024-04-25T13:16:08.207 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:16:08.207 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T13:16:08.371 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target.wants/ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968@.service. 2024-04-25T13:16:08.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:08 smithi037 ceph-mon[28835]: mkfs d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:08.856 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:08 smithi037 ceph-mon[28835]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:16:08.896 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:16:08.896 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T13:16:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:16:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:16:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T13:16:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:16:08.916 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-04-25T13:16:08.916 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-04-25T13:16:09.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:08 smithi037 bash[28817]: efe27ffd5861d324fde2a124a326e875bcd5cbb9aeec3002cc99838e7980e8fd 2024-04-25T13:16:09.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:08 smithi037 systemd[1]: Started Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: monmap epoch 1 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: last_changed 2024-04-25T13:15:55.450143+0000 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: min_mon_release 19 (squid) 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: election_strategy: 1 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: fsmap 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:16:09.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: mgrmap e1: no daemons active 2024-04-25T13:16:09.966 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:09 smithi037 ceph-mon[28835]: from='client.? 172.21.15.37:0/1506049970' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.542614s) 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T13:16:09.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.311+0000 7fe3f35cf700 1 Processor -- start 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f35cf700 1 -- start start 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f35cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f35cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec07c8b0 con 0x7fe3ec07de30 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f136b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f136b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51196/0 (socket says 172.21.15.37:51196) 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f136b700 1 -- 172.21.15.37:0/1069700778 learned_addr learned my addr 172.21.15.37:0/1069700778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.312+0000 7fe3f136b700 1 -- 172.21.15.37:0/1069700778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec07ca30 con 0x7fe3ec07de30 2024-04-25T13:16:09.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f136b700 1 --2- 172.21.15.37:0/1069700778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0098a0 tx=0x7fe3dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7914a6e0115ea131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3e3fff700 1 -- 172.21.15.37:0/1069700778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc004160 con 0x7fe3ec07de30 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3e3fff700 1 -- 172.21.15.37:0/1069700778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc004300 con 0x7fe3ec07de30 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3e3fff700 1 -- 172.21.15.37:0/1069700778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc0036a0 con 0x7fe3ec07de30 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1069700778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 msgr2=0x7fe3ec07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f35cf700 1 --2- 172.21.15.37:0/1069700778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0098a0 tx=0x7fe3dc009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1069700778 shutdown_connections 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f35cf700 1 --2- 172.21.15.37:0/1069700778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec07c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.313+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1069700778 >> 172.21.15.37:0/1069700778 conn(0x7fe3ec103bc0 msgr2=0x7fe3ec105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1069700778 shutdown_connections 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1069700778 wait complete. 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 Processor -- start 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 -- start start 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f35cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec07cf60 con 0x7fe3ec07de30 2024-04-25T13:16:09.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f136b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f136b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51198/0 (socket says 172.21.15.37:51198) 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.314+0000 7fe3f136b700 1 -- 172.21.15.37:0/1506049970 learned_addr learned my addr 172.21.15.37:0/1506049970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3f136b700 1 -- 172.21.15.37:0/1506049970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec19e360 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3f136b700 1 --2- 172.21.15.37:0/1506049970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc000c00 tx=0x7fe3dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc003db0 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc011630 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec19e5f0 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3dc011950 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec1a1290 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3dc011af0 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.315+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe3dc01c450 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.316+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec0640a0 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.318+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe3dc033080 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.392+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe3ec1a16d0 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.392+0000 7fe3e27fc700 1 -- 172.21.15.37:0/1506049970 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe3dc004260 con 0x7fe3ec07de30 2024-04-25T13:16:09.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 msgr2=0x7fe3ec19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 --2- 172.21.15.37:0/1506049970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc000c00 tx=0x7fe3dc004750 comp rx=0 tx=0).stop 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 shutdown_connections 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 --2- 172.21.15.37:0/1506049970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3ec07de30 0x7fe3ec19de20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 >> 172.21.15.37:0/1506049970 conn(0x7fe3ec103bc0 msgr2=0x7fe3ec104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 shutdown_connections 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:09.393+0000 7fe3f35cf700 1 -- 172.21.15.37:0/1506049970 wait complete. 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-04-25T13:16:09.978 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T13:16:11.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:10 smithi037 ceph-mon[28835]: from='client.? 172.21.15.37:0/2412070540' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:16:11.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:10 smithi037 ceph-mon[28835]: from='client.? 172.21.15.37:0/2412070540' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T13:16:11.050 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:11.050 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:16:11.050 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:11.050 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:16:11.051 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.403+0000 7fec99f95700 1 Processor -- start 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.403+0000 7fec99f95700 1 -- start start 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.403+0000 7fec99f95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.403+0000 7fec99f95700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec9407c8b0 con 0x7fec9407de30 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.403+0000 7fec937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39184/0 (socket says 172.21.15.37:39184) 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec937fe700 1 -- 172.21.15.37:0/3297271260 learned_addr learned my addr 172.21.15.37:0/3297271260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec937fe700 1 -- 172.21.15.37:0/3297271260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9407ca30 con 0x7fec9407de30 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec937fe700 1 --2- 172.21.15.37:0/3297271260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec84009e80 tx=0x7fec8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d00d6bf104eada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:11.052 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec927fc700 1 -- 172.21.15.37:0/3297271260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec84017070 con 0x7fec9407de30 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec927fc700 1 -- 172.21.15.37:0/3297271260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec84003a20 con 0x7fec9407de30 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.404+0000 7fec927fc700 1 -- 172.21.15.37:0/3297271260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec84003d20 con 0x7fec9407de30 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 -- 172.21.15.37:0/3297271260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 msgr2=0x7fec9407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 --2- 172.21.15.37:0/3297271260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fec84009e80 tx=0x7fec8400b330 comp rx=0 tx=0).stop 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 -- 172.21.15.37:0/3297271260 shutdown_connections 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 --2- 172.21.15.37:0/3297271260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9407c2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 -- 172.21.15.37:0/3297271260 >> 172.21.15.37:0/3297271260 conn(0x7fec94103c30 msgr2=0x7fec94106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 -- 172.21.15.37:0/3297271260 shutdown_connections 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 -- 172.21.15.37:0/3297271260 wait complete. 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.405+0000 7fec99f95700 1 Processor -- start 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec99f95700 1 -- start start 2024-04-25T13:16:11.053 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec99f95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec99f95700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec9407cf60 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec937fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39186/0 (socket says 172.21.15.37:39186) 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec937fe700 1 -- 172.21.15.37:0/2412070540 learned_addr learned my addr 172.21.15.37:0/2412070540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec937fe700 1 -- 172.21.15.37:0/2412070540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9419e370 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.406+0000 7fec937fe700 1 --2- 172.21.15.37:0/2412070540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec9407d350 tx=0x7fec84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec84022020 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec84018040 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9419e600 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec84017070 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941a12f0 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec84025020 con 0x7fec9407de30 2024-04-25T13:16:11.054 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.407+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec8400f050 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.408+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec940640a0 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.410+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fec8402c030 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.481+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fec9402e060 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.484+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fec8401e480 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.485+0000 7fec90ff9700 1 -- 172.21.15.37:0/2412070540 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec84035070 con 0x7fec9407de30 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 msgr2=0x7fec9419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 --2- 172.21.15.37:0/2412070540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fec9407d350 tx=0x7fec84004ad0 comp rx=0 tx=0).stop 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 shutdown_connections 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 --2- 172.21.15.37:0/2412070540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec9407de30 0x7fec9419de30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 >> 172.21.15.37:0/2412070540 conn(0x7fec94103c30 msgr2=0x7fec94105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 shutdown_connections 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:10.486+0000 7fec99f95700 1 -- 172.21.15.37:0/2412070540 wait complete. 2024-04-25T13:16:11.055 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff43093b700 1 Processor -- start 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff43093b700 1 -- start start 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff43093b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff43093b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c07be50 con 0x7ff42c1090a0 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39190/0 (socket says 172.21.15.37:39190) 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.468+0000 7ff42a59c700 1 -- 172.21.15.37:0/2115988931 learned_addr learned my addr 172.21.15.37:0/2115988931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.469+0000 7ff42a59c700 1 -- 172.21.15.37:0/2115988931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c10b9d0 con 0x7ff42c1090a0 2024-04-25T13:16:12.105 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.469+0000 7ff42a59c700 1 --2- 172.21.15.37:0/2115988931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098a0 tx=0x7ff41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f930be4e797cffef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.469+0000 7ff42959a700 1 -- 172.21.15.37:0/2115988931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff41c004160 con 0x7ff42c1090a0 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.469+0000 7ff42959a700 1 -- 172.21.15.37:0/2115988931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff41c004300 con 0x7ff42c1090a0 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.469+0000 7ff42959a700 1 -- 172.21.15.37:0/2115988931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff41c011af0 con 0x7ff42c1090a0 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 -- 172.21.15.37:0/2115988931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 msgr2=0x7ff42c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 --2- 172.21.15.37:0/2115988931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0098a0 tx=0x7ff41c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 -- 172.21.15.37:0/2115988931 shutdown_connections 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 --2- 172.21.15.37:0/2115988931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c10b490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 -- 172.21.15.37:0/2115988931 >> 172.21.15.37:0/2115988931 conn(0x7ff42c102a90 msgr2=0x7ff42c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 -- 172.21.15.37:0/2115988931 shutdown_connections 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 -- 172.21.15.37:0/2115988931 wait complete. 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.470+0000 7ff43093b700 1 Processor -- start 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff43093b700 1 -- start start 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff43093b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:12.106 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff43093b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10c920 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff42a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39194/0 (socket says 172.21.15.37:39194) 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff42a59c700 1 -- 172.21.15.37:0/839460866 learned_addr learned my addr 172.21.15.37:0/839460866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff42a59c700 1 -- 172.21.15.37:0/839460866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19c980 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff42a59c700 1 --2- 172.21.15.37:0/839460866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff41c003f40 tx=0x7ff41c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff41c0038b0 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff41c003a50 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19cc10 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.471+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff41c01a5c0 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.472+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c19f900 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.472+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff41c01ac50 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.472+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff41c019700 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.472+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c0640a0 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.475+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff41c0199a0 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.545+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff42c19fda0 con 0x7ff42c1090a0 2024-04-25T13:16:12.107 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.545+0000 7ff41b7fe700 1 -- 172.21.15.37:0/839460866 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff41c019b80 con 0x7ff42c1090a0 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 msgr2=0x7ff42c19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 --2- 172.21.15.37:0/839460866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff41c003f40 tx=0x7ff41c011890 comp rx=0 tx=0).stop 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 shutdown_connections 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 --2- 172.21.15.37:0/839460866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff42c1090a0 0x7ff42c19c440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 >> 172.21.15.37:0/839460866 conn(0x7ff42c102a90 msgr2=0x7ff42c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 shutdown_connections 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:11.546+0000 7ff43093b700 1 -- 172.21.15.37:0/839460866 wait complete. 2024-04-25T13:16:12.108 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-04-25T13:16:12.123 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:11 smithi037 ceph-mon[28835]: from='client.? 172.21.15.37:0/839460866' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:16:12.543 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 systemd[1]: Stopping Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:16:12.543 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a[28831]: 2024-04-25T13:16:12.359+0000 7fc3a27c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:16:12.543 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a[28831]: 2024-04-25T13:16:12.359+0000 7fc3a27c7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T13:16:12.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 podman[29080]: 2024-04-25 13:16:12.543989819 +0000 UTC m=+0.270586457 container died efe27ffd5861d324fde2a124a326e875bcd5cbb9aeec3002cc99838e7980e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T13:16:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 podman[29080]: 2024-04-25 13:16:12.908817598 +0000 UTC m=+0.635414262 container cleanup efe27ffd5861d324fde2a124a326e875bcd5cbb9aeec3002cc99838e7980e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:16:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:12 smithi037 bash[29080]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:16:13.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:13 smithi037 podman[29091]: 2024-04-25 13:16:13.159736787 +0000 UTC m=+0.610729202 container remove efe27ffd5861d324fde2a124a326e875bcd5cbb9aeec3002cc99838e7980e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T13:16:13.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:13 smithi037 systemd[1]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T13:16:13.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:13 smithi037 systemd[1]: Stopped Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:16:13.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:13 smithi037 systemd[1]: Starting Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:16:13.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:13 smithi037 podman[29185]: 2024-04-25 13:16:13.415043929 +0000 UTC m=+0.018564018 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:16:14.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:14 smithi037 podman[29185]: 2024-04-25 13:16:14.394223256 +0000 UTC m=+0.997743404 container create d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:16:15.083 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 podman[29185]: 2024-04-25 13:16:15.003912459 +0000 UTC m=+1.607432538 container init d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 podman[29185]: 2024-04-25 13:16:15.007202605 +0000 UTC m=+1.610722685 container start d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: pidfile_write: ignore empty --pid-file 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: load: jerasure load: lrc 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: RocksDB version: 7.9.2 2024-04-25T13:16:15.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Git sha 0 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: DB SUMMARY 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: DB Session ID: UTDYQS9ONWRO16FSUVQE 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: CURRENT file: CURRENT 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104880 ; 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.error_if_exists: 0 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.create_if_missing: 0 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.paranoid_checks: 1 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.env: 0x56362bddbcc0 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T13:16:15.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.info_log: 0x56362dc51be0 2024-04-25T13:16:15.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T13:16:15.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.statistics: (nil) 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.use_fsync: 0 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_log_file_size: 0 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T13:16:15.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_fallocate: 1 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.use_direct_reads: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.db_log_dir: 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.wal_dir: 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T13:16:15.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.write_buffer_manager: 0x56362dbe50e0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.unordered_write: 0 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T13:16:15.215 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.row_cache: None 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.wal_filter: None 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.two_write_queues: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.wal_compression: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.atomic_flush: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.log_readahead_size: 0 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T13:16:15.216 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_background_jobs: 2 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_background_compactions: -1 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_subcompactions: 1 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T13:16:15.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_open_files: -1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_background_flushes: -1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Compression algorithms supported: 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kZSTD supported: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kXpressCompression supported: 0 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kZlibCompression supported: 1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kSnappyCompression supported: 1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kLZ4Compression supported: 1 2024-04-25T13:16:15.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: kBZip2Compression supported: 0 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.merge_operator: 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_filter: None 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56362dc51dc0) 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T13:16:15.219 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x56362dc171f0 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-04-25T13:16:15.220 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-25T13:16:15.221 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression: NoCompression 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.num_levels: 7 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T13:16:15.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T13:16:15.223 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T13:16:15.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.inplace_update_support: 0 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T13:16:15.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.bloom_locality: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.max_successive_merges: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.ttl: 2592000 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enable_blob_files: false 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.min_blob_size: 0 2024-04-25T13:16:15.226 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 30430346-9fc0-4248-bb3e-6f1cb329e1b0 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050975039321, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050975040406, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97910, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90254, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714050975, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "30430346-9fc0-4248-bb3e-6f1cb329e1b0", "db_session_id": "UTDYQS9ONWRO16FSUVQE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714050975040500, "job": 1, "event": "recovery_finished"} 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T13:16:15.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56362dd2c000 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: DB pointer 0x56362dd16000 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:16:15.228 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 99.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 99.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T13:16:15.229 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 28.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 28.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x56362dc171f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???) e1 preinit fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).mds e1 new map 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).mds e1 print_map 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-04-25T13:16:15.230 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 bash[29185]: d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 systemd[1]: Started Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: monmap epoch 1 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: last_changed 2024-04-25T13:15:55.450143+0000 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: min_mon_release 19 (squid) 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: election_strategy: 1 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:16:15.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: fsmap 2024-04-25T13:16:15.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T13:16:15.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:15 smithi037 ceph-mon[29204]: mgrmap e1: no daemons active 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a868f700 1 Processor -- start 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a868f700 1 -- start start 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a868f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a868f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a007c8b0 con 0x7fc9a007de30 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a642b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a642b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39204/0 (socket says 172.21.15.37:39204) 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.588+0000 7fc9a642b700 1 -- 172.21.15.37:0/3429041399 learned_addr learned my addr 172.21.15.37:0/3429041399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.589+0000 7fc9a642b700 1 -- 172.21.15.37:0/3429041399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a007ca30 con 0x7fc9a007de30 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.589+0000 7fc9a642b700 1 --2- 172.21.15.37:0/3429041399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098a0 tx=0x7fc99c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112ff0d08016a10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.589+0000 7fc9a5429700 1 -- 172.21.15.37:0/3429041399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc99c004160 con 0x7fc9a007de30 2024-04-25T13:16:16.158 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.589+0000 7fc9a5429700 1 -- 172.21.15.37:0/3429041399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc99c004300 con 0x7fc9a007de30 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.589+0000 7fc9a5429700 1 -- 172.21.15.37:0/3429041399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc99c011af0 con 0x7fc9a007de30 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- 172.21.15.37:0/3429041399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 msgr2=0x7fc9a007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 --2- 172.21.15.37:0/3429041399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098a0 tx=0x7fc99c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- 172.21.15.37:0/3429041399 shutdown_connections 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 --2- 172.21.15.37:0/3429041399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a007c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- 172.21.15.37:0/3429041399 >> 172.21.15.37:0/3429041399 conn(0x7fc9a0103bc0 msgr2=0x7fc9a0105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- 172.21.15.37:0/3429041399 shutdown_connections 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- 172.21.15.37:0/3429041399 wait complete. 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 Processor -- start 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.590+0000 7fc9a868f700 1 -- start start 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a868f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a868f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a010eaf0 con 0x7fc9a007de30 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a642b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a642b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39208/0 (socket says 172.21.15.37:39208) 2024-04-25T13:16:16.159 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a642b700 1 -- 172.21.15.37:0/1813950503 learned_addr learned my addr 172.21.15.37:0/1813950503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a642b700 1 -- 172.21.15.37:0/1813950503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a019eb80 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a642b700 1 --2- 172.21.15.37:0/1813950503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011ac0 tx=0x7fc99c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc99c003b20 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc99c01a8c0 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a019ee10 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc99c0248d0 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.591+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a01a1b00 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.592+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc99c022020 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.592+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc99c0193c0 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.592+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a00640a0 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.595+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc99c042e50 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.665+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fc9a01a1cd0 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.667+0000 7fc9937fe700 1 -- 172.21.15.37:0/1813950503 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fc99c031020 con 0x7fc9a007de30 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 msgr2=0x7fc9a019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:16.160 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 --2- 172.21.15.37:0/1813950503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc99c011ac0 tx=0x7fc99c011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 shutdown_connections 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 --2- 172.21.15.37:0/1813950503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9a007de30 0x7fc9a019e640 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 >> 172.21.15.37:0/1813950503 conn(0x7fc9a0103bc0 msgr2=0x7fc9a0104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 shutdown_connections 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:15.668+0000 7fc9a868f700 1 -- 172.21.15.37:0/1813950503 wait complete. 2024-04-25T13:16:16.161 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T13:16:16.176 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:16:16.176 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-04-25T13:16:16.176 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T13:16:16.176 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T13:16:16.425 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:16:16.426 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T13:16:16.577 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968.target.wants/ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d766aeb0-0305-11ef-bc93-c7b262605968@.service. 2024-04-25T13:16:16.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:16 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1813950503' entity='client.admin' 2024-04-25T13:16:16.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:16 smithi037 podman[29423]: 2024-04-25 13:16:16.722507809 +0000 UTC m=+0.018487075 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T13:16:17.128 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:16:17.129 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T13:16:17.136 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:16:17.136 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:16:17.136 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T13:16:17.136 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T13:16:17.145 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:16:17.145 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T13:16:17.152 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:16:17.153 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:16:17.153 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T13:16:17.153 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T13:16:17.153 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-04-25T13:16:17.153 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-04-25T13:16:17.174 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 podman[29423]: 2024-04-25 13:16:17.014633396 +0000 UTC m=+0.310612638 container init 4773bce144a732077bf3e4a50fc76decdd0c976aea1960e17c12d6b8ac191789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:16:17.174 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 podman[29423]: 2024-04-25 13:16:17.017270229 +0000 UTC m=+0.313249469 container start 4773bce144a732077bf3e4a50fc76decdd0c976aea1960e17c12d6b8ac191789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:16:17.174 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 bash[29423]: 4773bce144a732077bf3e4a50fc76decdd0c976aea1960e17c12d6b8ac191789 2024-04-25T13:16:17.175 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 systemd[1]: Started Ceph mgr.x for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:16:17.497 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:17.174+0000 7fca7af9f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:16:17.497 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:17.369+0000 7fca7af9f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:16:17.497 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:17.475+0000 7fca7af9f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:16:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/793214019' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:16:18.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:17.804+0000 7fca7af9f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:16:18.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:17.895+0000 7fca7af9f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:16:18.750 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:16:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:16:18.752 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:16:18.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:16:18.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:16:08.852203+0000", 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.820+0000 7f6e03cc0700 1 Processor -- start 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.820+0000 7f6e03cc0700 1 -- start start 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.820+0000 7f6e03cc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.820+0000 7f6e03cc0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc107ec0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.821+0000 7f6e01a5c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:18.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.821+0000 7f6e01a5c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39230/0 (socket says 172.21.15.37:39230) 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.821+0000 7f6e01a5c700 1 -- 172.21.15.37:0/1694978062 learned_addr learned my addr 172.21.15.37:0/1694978062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.821+0000 7f6e01a5c700 1 -- 172.21.15.37:0/1694978062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc1086d0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.821+0000 7f6e01a5c700 1 --2- 172.21.15.37:0/1694978062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6df800bb40 tx=0x7f6df800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9321f555685e0dae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e00a5a700 1 -- 172.21.15.37:0/1694978062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df800dbe0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e00a5a700 1 -- 172.21.15.37:0/1694978062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6df80044c0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e00a5a700 1 -- 172.21.15.37:0/1694978062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8010470 con 0x7f6dfc1074d0 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 -- 172.21.15.37:0/1694978062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 msgr2=0x7f6dfc1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 --2- 172.21.15.37:0/1694978062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6df800bb40 tx=0x7f6df800be50 comp rx=0 tx=0).stop 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 -- 172.21.15.37:0/1694978062 shutdown_connections 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 --2- 172.21.15.37:0/1694978062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc1078f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 -- 172.21.15.37:0/1694978062 >> 172.21.15.37:0/1694978062 conn(0x7f6dfc102ab0 msgr2=0x7f6dfc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 -- 172.21.15.37:0/1694978062 shutdown_connections 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.822+0000 7f6e03cc0700 1 -- 172.21.15.37:0/1694978062 wait complete. 2024-04-25T13:16:18.764 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e03cc0700 1 Processor -- start 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e03cc0700 1 -- start start 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e03cc0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e03cc0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc062e80 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e01a5c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e01a5c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39244/0 (socket says 172.21.15.37:39244) 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.823+0000 7f6e01a5c700 1 -- 172.21.15.37:0/793214019 learned_addr learned my addr 172.21.15.37:0/793214019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6e01a5c700 1 -- 172.21.15.37:0/793214019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc19a790 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6e01a5c700 1 --2- 172.21.15.37:0/793214019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b340 tx=0x7f6df8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8027070 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6df8010bb0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc19aa20 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df8022ac0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc19d710 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6df802a070 con 0x7f6dfc1074d0 2024-04-25T13:16:18.765 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.824+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6df801f070 con 0x7f6dfc1074d0 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.825+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc0640a0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.827+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6df8031030 con 0x7f6dfc1074d0 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.901+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6dfc1937b0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.902+0000 7f6deeffd700 1 -- 172.21.15.37:0/793214019 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6dfc1937b0 con 0x7f6dfc1074d0 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 msgr2=0x7f6dfc19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 --2- 172.21.15.37:0/793214019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b340 tx=0x7f6df8003dc0 comp rx=0 tx=0).stop 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 shutdown_connections 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 --2- 172.21.15.37:0/793214019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6dfc1074d0 0x7f6dfc19a250 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 >> 172.21.15.37:0/793214019 conn(0x7f6dfc102ab0 msgr2=0x7f6dfc190890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 shutdown_connections 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:17.903+0000 7f6e03cc0700 1 -- 172.21.15.37:0/793214019 wait complete. 2024-04-25T13:16:18.766 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-04-25T13:16:19.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:19.176+0000 7fca7af9f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:16:19.380 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:19.355+0000 7fca7af9f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:16:19.843 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:19.536+0000 7fca7af9f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:16:19.843 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:19.842+0000 7fca7af9f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:16:20.100 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:19.939+0000 7fca7af9f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:16:20.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:20.150+0000 7fca7af9f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:16:20.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:20.352+0000 7fca7af9f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:16:21.132 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:20.989+0000 7fca7af9f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:16:21.444 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:21 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3785240307' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:16:21.768 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:21.442+0000 7fca7af9f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:16:21.768 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:21.542+0000 7fca7af9f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:16:21.768 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:21.751+0000 7fca7af9f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:16:22.324 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:16:22.325 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:22.326 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:16:22.326 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:16:22.326 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:16:22.334 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:16:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:16:08.852203+0000", 2024-04-25T13:16:22.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f286ab53700 1 Processor -- start 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f286ab53700 1 -- start start 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f286ab53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f286ab53700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f286407be30 con 0x7f2864106e70 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f28688ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f28688ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49738/0 (socket says 172.21.15.37:49738) 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f28688ef700 1 -- 172.21.15.37:0/1524073234 learned_addr learned my addr 172.21.15.37:0/1524073234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:22.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.234+0000 7f28688ef700 1 -- 172.21.15.37:0/1524073234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28641077d0 con 0x7f2864106e70 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f28688ef700 1 --2- 172.21.15.37:0/1524073234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28580098a0 tx=0x7f2858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cfe4b42423957a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f28637fe700 1 -- 172.21.15.37:0/1524073234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858004160 con 0x7f2864106e70 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f28637fe700 1 -- 172.21.15.37:0/1524073234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2858004300 con 0x7f2864106e70 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f28637fe700 1 -- 172.21.15.37:0/1524073234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858011af0 con 0x7f2864106e70 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 -- 172.21.15.37:0/1524073234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 msgr2=0x7f2864107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 --2- 172.21.15.37:0/1524073234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28580098a0 tx=0x7f2858009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 -- 172.21.15.37:0/1524073234 shutdown_connections 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 --2- 172.21.15.37:0/1524073234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f2864107290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 -- 172.21.15.37:0/1524073234 >> 172.21.15.37:0/1524073234 conn(0x7f2864102a70 msgr2=0x7f2864104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.235+0000 7f286ab53700 1 -- 172.21.15.37:0/1524073234 shutdown_connections 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f286ab53700 1 -- 172.21.15.37:0/1524073234 wait complete. 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f286ab53700 1 Processor -- start 2024-04-25T13:16:22.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f286ab53700 1 -- start start 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f286ab53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f286ab53700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2864108790 con 0x7f2864106e70 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f28688ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f28688ef700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49740/0 (socket says 172.21.15.37:49740) 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.236+0000 7f28688ef700 1 -- 172.21.15.37:0/3785240307 learned_addr learned my addr 172.21.15.37:0/3785240307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f28688ef700 1 -- 172.21.15.37:0/3785240307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286419ec30 con 0x7f2864106e70 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f28688ef700 1 --2- 172.21.15.37:0/3785240307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fb0 tx=0x7f2858011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28580036a0 con 0x7f2864106e70 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f285801a430 con 0x7f2864106e70 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286419eec0 con 0x7f2864106e70 2024-04-25T13:16:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858019550 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641a1bb0 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2858019be0 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.237+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2858024730 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.238+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28640640a0 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.240+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2858043e50 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.314+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f286402e090 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.314+0000 7f2861ffb700 1 -- 172.21.15.37:0/3785240307 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2858032030 con 0x7f2864106e70 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.315+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 msgr2=0x7f286419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.315+0000 7f286ab53700 1 --2- 172.21.15.37:0/3785240307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fb0 tx=0x7f2858011940 comp rx=0 tx=0).stop 2024-04-25T13:16:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.315+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 shutdown_connections 2024-04-25T13:16:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.315+0000 7f286ab53700 1 --2- 172.21.15.37:0/3785240307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2864106e70 0x7f286419e6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.315+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 >> 172.21.15.37:0/3785240307 conn(0x7f2864102a70 msgr2=0x7f28641953d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.316+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 shutdown_connections 2024-04-25T13:16:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:21.316+0000 7f286ab53700 1 -- 172.21.15.37:0/3785240307 wait complete. 2024-04-25T13:16:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-04-25T13:16:22.350 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:22.298+0000 7fca7af9f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:16:22.647 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:22.395+0000 7fca7af9f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:16:23.002 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:22.646+0000 7fca7af9f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:16:23.002 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:22.784+0000 7fca7af9f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:16:23.002 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:22.882+0000 7fca7af9f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:16:23.002 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:23 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:23.001+0000 7fca7af9f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:16:23.600 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:23 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:23.313+0000 7fca7af9f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:16:24.221 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:23 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:23.943+0000 7fca7af9f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:16:24.221 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:24.166+0000 7fca7af9f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:16:24.557 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: Activating manager daemon x 2024-04-25T13:16:24.557 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: mgrmap e2: x(active, starting, since 0.0030831s) 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: Manager daemon x is now available 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' 2024-04-25T13:16:24.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' 2024-04-25T13:16:24.559 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:24 smithi037 ceph-mon[29204]: from='mgr.14100 172.21.15.37:0/2338654110' entity='mgr.x' 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:16:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:16:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:16:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:25.434 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:16:08.852203+0000", 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.762+0000 7fe9165ea700 1 Processor -- start 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.762+0000 7fe9165ea700 1 -- start start 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.762+0000 7fe9165ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe9165ea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe910063d60 con 0x7fe910109040 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49810/0 (socket says 172.21.15.37:49810) 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90ffff700 1 -- 172.21.15.37:0/1671355320 learned_addr learned my addr 172.21.15.37:0/1671355320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:25.435 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90ffff700 1 -- 172.21.15.37:0/1671355320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91010b970 con 0x7fe910109040 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90ffff700 1 --2- 172.21.15.37:0/1671355320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9000098a0 tx=0x7fe900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d1af33a745ae05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.763+0000 7fe90effd700 1 -- 172.21.15.37:0/1671355320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900004160 con 0x7fe910109040 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe90effd700 1 -- 172.21.15.37:0/1671355320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe900004300 con 0x7fe910109040 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe90effd700 1 -- 172.21.15.37:0/1671355320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900011af0 con 0x7fe910109040 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 -- 172.21.15.37:0/1671355320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 msgr2=0x7fe91010b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 --2- 172.21.15.37:0/1671355320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe9000098a0 tx=0x7fe900009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 -- 172.21.15.37:0/1671355320 shutdown_connections 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 --2- 172.21.15.37:0/1671355320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe91010b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 -- 172.21.15.37:0/1671355320 >> 172.21.15.37:0/1671355320 conn(0x7fe910102ab0 msgr2=0x7fe910104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 -- 172.21.15.37:0/1671355320 shutdown_connections 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.764+0000 7fe9165ea700 1 -- 172.21.15.37:0/1671355320 wait complete. 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe9165ea700 1 Processor -- start 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe9165ea700 1 -- start start 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe9165ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:25.436 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe9165ea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010c8d0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49814/0 (socket says 172.21.15.37:49814) 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.765+0000 7fe90ffff700 1 -- 172.21.15.37:0/3826269565 learned_addr learned my addr 172.21.15.37:0/3826269565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe90ffff700 1 -- 172.21.15.37:0/3826269565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe910108e00 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe90ffff700 1 --2- 172.21.15.37:0/3826269565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe900011ac0 tx=0x7fe900011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900003b20 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe910106ff0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe90001a8c0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9000248d0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.766+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe910107530 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.767+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fe900022020 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.767+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9100640a0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.767+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe90004d290 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.769+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe90002d4a0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.842+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9101077b0 con 0x7fe910109040 2024-04-25T13:16:25.437 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.842+0000 7fe90d7fa700 1 -- 172.21.15.37:0/3826269565 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe90001aae0 con 0x7fe910109040 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.843+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 msgr2=0x7fe9101088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.843+0000 7fe9165ea700 1 --2- 172.21.15.37:0/3826269565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe900011ac0 tx=0x7fe900011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.843+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 shutdown_connections 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.843+0000 7fe9165ea700 1 --2- 172.21.15.37:0/3826269565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe910109040 0x7fe9101088c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.843+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 >> 172.21.15.37:0/3826269565 conn(0x7fe910102ab0 msgr2=0x7fe910103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.844+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 shutdown_connections 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:24.844+0000 7fe9165ea700 1 -- 172.21.15.37:0/3826269565 wait complete. 2024-04-25T13:16:25.438 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-04-25T13:16:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:25 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3826269565' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:16:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:25 smithi037 ceph-mon[29204]: mgrmap e3: x(active, since 1.00698s) 2024-04-25T13:16:27.533 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:27 smithi037 ceph-mon[29204]: mgrmap e4: x(active, since 2s) 2024-04-25T13:16:28.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:28 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1483844239' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T13:16:28.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T13:16:28.779 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T13:16:28.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T13:16:28.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T13:16:08.852203+0000", 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff89b3d8700 1 Processor -- start 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff89b3d8700 1 -- start start 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff89b3d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff89b3d8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff89407c800 con 0x7ff89407de10 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff899174700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff899174700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49828/0 (socket says 172.21.15.37:49828) 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.846+0000 7ff899174700 1 -- 172.21.15.37:0/4108313978 learned_addr learned my addr 172.21.15.37:0/4108313978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff899174700 1 -- 172.21.15.37:0/4108313978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89407c980 con 0x7ff89407de10 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff899174700 1 --2- 172.21.15.37:0/4108313978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098a0 tx=0x7ff890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ac187af246c1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff88bfff700 1 -- 172.21.15.37:0/4108313978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890004160 con 0x7ff89407de10 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff88bfff700 1 -- 172.21.15.37:0/4108313978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff890004300 con 0x7ff89407de10 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff88bfff700 1 -- 172.21.15.37:0/4108313978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890011af0 con 0x7ff89407de10 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff89b3d8700 1 -- 172.21.15.37:0/4108313978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 msgr2=0x7ff89407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.847+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/4108313978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8900098a0 tx=0x7ff890009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- 172.21.15.37:0/4108313978 shutdown_connections 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/4108313978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89407c2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- 172.21.15.37:0/4108313978 >> 172.21.15.37:0/4108313978 conn(0x7ff894103bc0 msgr2=0x7ff894105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- 172.21.15.37:0/4108313978 shutdown_connections 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- 172.21.15.37:0/4108313978 wait complete. 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 Processor -- start 2024-04-25T13:16:28.791 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- start start 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.848+0000 7ff89b3d8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff89410eab0 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff899174700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff899174700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49832/0 (socket says 172.21.15.37:49832) 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff899174700 1 -- 172.21.15.37:0/1483844239 learned_addr learned my addr 172.21.15.37:0/1483844239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff899174700 1 -- 172.21.15.37:0/1483844239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89419ebd0 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff899174700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff890011ac0 tx=0x7ff890011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff890003b20 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff89001a8c0 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8900248d0 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89419ee60 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.849+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8941a1b50 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.850+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff890022020 con 0x7ff89407de10 2024-04-25T13:16:28.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.850+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8940640a0 con 0x7ff89407de10 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.850+0000 7ff88a7fc700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 0x7ff88003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.850+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff89004d4c0 con 0x7ff89407de10 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.850+0000 7ff898973700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 0x7ff88003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.851+0000 7ff898973700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 0x7ff88003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff884006fd0 tx=0x7ff884006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:27.856+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff89001ab20 con 0x7ff89407de10 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.148+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff8941a1d20 con 0x7ff89407de10 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.148+0000 7ff88a7fc700 1 -- 172.21.15.37:0/1483844239 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff89004d220 con 0x7ff89407de10 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 msgr2=0x7ff88003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 0x7ff88003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff884006fd0 tx=0x7ff884006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 msgr2=0x7ff89419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff890011ac0 tx=0x7ff890011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 shutdown_connections 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7ff8800395b0 0x7ff88003ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 --2- 172.21.15.37:0/1483844239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff89407de10 0x7ff89419e690 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:28.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.150+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 >> 172.21.15.37:0/1483844239 conn(0x7ff894103bc0 msgr2=0x7ff894104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:28.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.151+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 shutdown_connections 2024-04-25T13:16:28.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:28.151+0000 7ff89b3d8700 1 -- 172.21.15.37:0/1483844239 wait complete. 2024-04-25T13:16:28.794 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-04-25T13:16:29.824 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:29 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2664381035' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T13:16:29.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:29.929 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T13:16:29.930 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.200+0000 7f65fe172700 1 Processor -- start 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.200+0000 7f65fe172700 1 -- start start 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65fe172700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65fe172700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8063d20 con 0x7f65f8106e70 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49846/0 (socket says 172.21.15.37:49846) 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f77fe700 1 -- 172.21.15.37:0/781021122 learned_addr learned my addr 172.21.15.37:0/781021122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f77fe700 1 -- 172.21.15.37:0/781021122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f81077d0 con 0x7f65f8106e70 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f77fe700 1 --2- 172.21.15.37:0/781021122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098a0 tx=0x7f65e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc45873a8075cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f6ffd700 1 -- 172.21.15.37:0/781021122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e8004160 con 0x7f65f8106e70 2024-04-25T13:16:29.931 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.201+0000 7f65f6ffd700 1 -- 172.21.15.37:0/781021122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65e8004300 con 0x7f65f8106e70 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65f6ffd700 1 -- 172.21.15.37:0/781021122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e8011af0 con 0x7f65f8106e70 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 -- 172.21.15.37:0/781021122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 msgr2=0x7f65f8107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 --2- 172.21.15.37:0/781021122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f65e80098a0 tx=0x7f65e8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 -- 172.21.15.37:0/781021122 shutdown_connections 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 --2- 172.21.15.37:0/781021122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f8107290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 -- 172.21.15.37:0/781021122 >> 172.21.15.37:0/781021122 conn(0x7f65f8102ab0 msgr2=0x7f65f8104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 -- 172.21.15.37:0/781021122 shutdown_connections 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.202+0000 7f65fe172700 1 -- 172.21.15.37:0/781021122 wait complete. 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65fe172700 1 Processor -- start 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65fe172700 1 -- start start 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65fe172700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65fe172700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8108790 con 0x7f65f8106e70 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:29.932 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49862/0 (socket says 172.21.15.37:49862) 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.203+0000 7f65f77fe700 1 -- 172.21.15.37:0/2664381035 learned_addr learned my addr 172.21.15.37:0/2664381035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65f77fe700 1 -- 172.21.15.37:0/2664381035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f819ebe0 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65f77fe700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65e8003f40 tx=0x7f65e8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e80038b0 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f65e8003a50 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f819ee70 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65e801a5c0 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.204+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81a1b60 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.205+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f65e801a870 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.205+0000 7f65f57fa700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 0x7f65e403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.205+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65e804da30 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.205+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f80640a0 con 0x7f65f8106e70 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.205+0000 7f65eeffd700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 0x7f65e403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:29.933 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.206+0000 7f65eeffd700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 0x7f65e403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.211+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65e8019c70 con 0x7f65f8106e70 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.418+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f65f802e0d0 con 0x7f65f8106e70 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.419+0000 7f65f57fa700 1 -- 172.21.15.37:0/2664381035 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f65e802c9c0 con 0x7f65f8106e70 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.421+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 msgr2=0x7f65e403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.421+0000 7f65fe172700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 0x7f65e403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.421+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 msgr2=0x7f65f819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.421+0000 7f65fe172700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65e8003f40 tx=0x7f65e8011890 comp rx=0 tx=0).stop 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 shutdown_connections 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f65e4039560 0x7f65e403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 --2- 172.21.15.37:0/2664381035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65f8106e70 0x7f65f819e6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 >> 172.21.15.37:0/2664381035 conn(0x7f65f8102ab0 msgr2=0x7f65f8195280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 shutdown_connections 2024-04-25T13:16:29.934 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:29.422+0000 7f65fe172700 1 -- 172.21.15.37:0/2664381035 wait complete. 2024-04-25T13:16:29.935 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-04-25T13:16:31.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:30 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2071289755' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T13:16:31.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setuser ceph since I am not root 2024-04-25T13:16:31.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setgroup ceph since I am not root 2024-04-25T13:16:31.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:30.883+0000 7f50425f6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:16:31.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:31.076+0000 7f50425f6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:16:31.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:31.182+0000 7f50425f6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:16:31.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.375+0000 7efd13a16700 1 Processor -- start 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd13a16700 1 -- start start 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd13a16700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd13a16700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c107290 con 0x7efd0c1068a0 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd117b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd117b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41070/0 (socket says 172.21.15.37:41070) 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd117b2700 1 -- 172.21.15.37:0/163409071 learned_addr learned my addr 172.21.15.37:0/163409071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd117b2700 1 -- 172.21.15.37:0/163409071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c1098d0 con 0x7efd0c1068a0 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.376+0000 7efd117b2700 1 --2- 172.21.15.37:0/163409071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd080098a0 tx=0x7efd08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e65b4aad50364b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd03fff700 1 -- 172.21.15.37:0/163409071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd08004160 con 0x7efd0c1068a0 2024-04-25T13:16:31.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd03fff700 1 -- 172.21.15.37:0/163409071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd08004300 con 0x7efd0c1068a0 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd03fff700 1 -- 172.21.15.37:0/163409071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd08011af0 con 0x7efd0c1068a0 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd13a16700 1 -- 172.21.15.37:0/163409071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 msgr2=0x7efd0c106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd13a16700 1 --2- 172.21.15.37:0/163409071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd080098a0 tx=0x7efd08009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd13a16700 1 -- 172.21.15.37:0/163409071 shutdown_connections 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd13a16700 1 --2- 172.21.15.37:0/163409071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c106cc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.377+0000 7efd13a16700 1 -- 172.21.15.37:0/163409071 >> 172.21.15.37:0/163409071 conn(0x7efd0c1021d0 msgr2=0x7efd0c104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 -- 172.21.15.37:0/163409071 shutdown_connections 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 -- 172.21.15.37:0/163409071 wait complete. 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 Processor -- start 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 -- start start 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:31.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd13a16700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c107410 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd117b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd117b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41074/0 (socket says 172.21.15.37:41074) 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.378+0000 7efd117b2700 1 -- 172.21.15.37:0/2071289755 learned_addr learned my addr 172.21.15.37:0/2071289755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd117b2700 1 -- 172.21.15.37:0/2071289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c19ca60 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd117b2700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd08009fb0 tx=0x7efd08011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd080036a0 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c19ccf0 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efd0801a430 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd08019550 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.379+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c19f9e0 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.380+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7efd08019850 con 0x7efd0c1068a0 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.380+0000 7efd027fc700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 0x7efcf803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:31.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.380+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd0804e880 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.380+0000 7efd10fb1700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 0x7efcf803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.380+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c0640a0 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.381+0000 7efd10fb1700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 0x7efcf803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006fd0 tx=0x7efcfc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.386+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd08023c90 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.655+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7efd0c1079a0 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.706+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7efd0804e2a0 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.708+0000 7efd027fc700 1 -- 172.21.15.37:0/2071289755 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7efd0802d960 con 0x7efd0c1068a0 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.709+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 msgr2=0x7efcf803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.709+0000 7efd13a16700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 0x7efcf803b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcfc006fd0 tx=0x7efcfc006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.709+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 msgr2=0x7efd0c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.709+0000 7efd13a16700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efd08009fb0 tx=0x7efd08011940 comp rx=0 tx=0).stop 2024-04-25T13:16:31.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 shutdown_connections 2024-04-25T13:16:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7efcf8039510 0x7efcf803b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 --2- 172.21.15.37:0/2071289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd0c1068a0 0x7efd0c19c520 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 >> 172.21.15.37:0/2071289755 conn(0x7efd0c1021d0 msgr2=0x7efd0c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 shutdown_connections 2024-04-25T13:16:31.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:30.710+0000 7efd13a16700 1 -- 172.21.15.37:0/2071289755 wait complete. 2024-04-25T13:16:31.711 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:31.506+0000 7f50425f6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:16:31.712 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:31.594+0000 7f50425f6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:16:32.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2071289755' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T13:16:32.061 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:31 smithi037 ceph-mon[29204]: mgrmap e5: x(active, since 6s) 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:16:32.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.854+0000 7f1e2dcfd700 1 Processor -- start 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.854+0000 7f1e2dcfd700 1 -- start start 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e2dcfd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e2dcfd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28063d20 con 0x7f1e28106e70 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e277fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e277fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41092/0 (socket says 172.21.15.37:41092) 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e277fe700 1 -- 172.21.15.37:0/4095135200 learned_addr learned my addr 172.21.15.37:0/4095135200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e277fe700 1 -- 172.21.15.37:0/4095135200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e281077d0 con 0x7f1e28106e70 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e277fe700 1 --2- 172.21.15.37:0/4095135200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e180098a0 tx=0x7f1e18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec1ffac69eaf7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e267fc700 1 -- 172.21.15.37:0/4095135200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e18004160 con 0x7f1e28106e70 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e267fc700 1 -- 172.21.15.37:0/4095135200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1e18004300 con 0x7f1e28106e70 2024-04-25T13:16:32.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.855+0000 7f1e267fc700 1 -- 172.21.15.37:0/4095135200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e18011af0 con 0x7f1e28106e70 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/4095135200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 msgr2=0x7f1e28107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/4095135200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e180098a0 tx=0x7f1e18009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/4095135200 shutdown_connections 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/4095135200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e28107290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/4095135200 >> 172.21.15.37:0/4095135200 conn(0x7f1e28102ab0 msgr2=0x7f1e28104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/4095135200 shutdown_connections 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.856+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/4095135200 wait complete. 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e2dcfd700 1 Processor -- start 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e2dcfd700 1 -- start start 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e2dcfd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:32.738 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e2dcfd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28108790 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e277fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e277fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41106/0 (socket says 172.21.15.37:41106) 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.857+0000 7f1e277fe700 1 -- 172.21.15.37:0/1968304679 learned_addr learned my addr 172.21.15.37:0/1968304679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e277fe700 1 -- 172.21.15.37:0/1968304679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2819ebe0 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e277fe700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e18011ac0 tx=0x7f1e18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e18003b20 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1e1801a8c0 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2819ee70 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e180248d0 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.858+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e281a1b60 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f1e18022020 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e280640a0 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e24ff9700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 0x7f1e1003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e1804d4f0 con 0x7f1e28106e70 2024-04-25T13:16:32.739 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e26ffd700 1 -- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 msgr2=0x7f1e1003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.859+0000 7f1e26ffd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 0x7f1e1003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:31.864+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e1801ab20 con 0x7f1e28106e70 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.060+0000 7f1e26ffd700 1 -- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 msgr2=0x7f1e1003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.060+0000 7f1e26ffd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 0x7f1e1003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.180+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1e2802e1f0 con 0x7f1e28106e70 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.181+0000 7f1e24ff9700 1 -- 172.21.15.37:0/1968304679 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1e1804d250 con 0x7f1e28106e70 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 msgr2=0x7f1e1003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 0x7f1e1003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 msgr2=0x7f1e2819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e18011ac0 tx=0x7f1e18011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 shutdown_connections 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f1e10039600 0x7f1e1003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 --2- 172.21.15.37:0/1968304679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e28106e70 0x7f1e2819e6a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 >> 172.21.15.37:0/1968304679 conn(0x7f1e28102ab0 msgr2=0x7f1e28195280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:32.740 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 shutdown_connections 2024-04-25T13:16:32.741 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:32.183+0000 7f1e2dcfd700 1 -- 172.21.15.37:0/1968304679 wait complete. 2024-04-25T13:16:32.741 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-25T13:16:32.741 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-04-25T13:16:32.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:32 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1968304679' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:16:33.159 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:32 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:32.860+0000 7f50425f6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:16:33.160 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:33.039+0000 7f50425f6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:16:33.519 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:33.219+0000 7f50425f6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:16:33.520 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:33.519+0000 7f50425f6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:16:33.786 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:33.615+0000 7f50425f6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:16:34.100 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:33.824+0000 7f50425f6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:16:34.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:34 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:34.024+0000 7f50425f6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:16:34.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:34 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:34.646+0000 7f50425f6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:16:35.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:35.083+0000 7f50425f6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:16:35.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:35.182+0000 7f50425f6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:16:35.850 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:35.392+0000 7f50425f6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:16:36.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:35.957+0000 7f50425f6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:16:36.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.052+0000 7f50425f6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:16:36.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.306+0000 7f50425f6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:16:36.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.445+0000 7f50425f6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:16:36.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.544+0000 7f50425f6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:16:36.978 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.664+0000 7f50425f6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:16:36.978 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:36.978+0000 7f50425f6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:16:37.885 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:37.606+0000 7f50425f6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:16:37.885 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:37.832+0000 7f50425f6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: Active manager daemon x restarted 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: Activating manager daemon x 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: mgrmap e6: x(active, starting, since 0.00342053s) 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: Manager daemon x is now available 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:38.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:38.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:38.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:37 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:39.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: Found migration_current of "None". Setting to last migration. 2024-04-25T13:16:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:16:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:16:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:38 smithi037 ceph-mon[29204]: mgrmap e7: x(active, since 1.0053s) 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.179+0000 7f7f4a3f0700 1 Processor -- start 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.179+0000 7f7f4a3f0700 1 -- start start 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.179+0000 7f7f4a3f0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.179+0000 7f7f4a3f0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44107ec0 con 0x7f7f441074d0 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f43fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f43fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41116/0 (socket says 172.21.15.37:41116) 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f43fff700 1 -- 172.21.15.37:0/4236404187 learned_addr learned my addr 172.21.15.37:0/4236404187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:39.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f43fff700 1 -- 172.21.15.37:0/4236404187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f441086d0 con 0x7f7f441074d0 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f43fff700 1 --2- 172.21.15.37:0/4236404187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66545906be3d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f42ffd700 1 -- 172.21.15.37:0/4236404187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38004160 con 0x7f7f441074d0 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f42ffd700 1 -- 172.21.15.37:0/4236404187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f38004300 con 0x7f7f441074d0 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.180+0000 7f7f42ffd700 1 -- 172.21.15.37:0/4236404187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38011af0 con 0x7f7f441074d0 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/4236404187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 msgr2=0x7f7f441078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/4236404187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/4236404187 shutdown_connections 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/4236404187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f441078f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/4236404187 >> 172.21.15.37:0/4236404187 conn(0x7f7f44102ab0 msgr2=0x7f7f44104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/4236404187 shutdown_connections 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.181+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/4236404187 wait complete. 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f4a3f0700 1 Processor -- start 2024-04-25T13:16:39.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f4a3f0700 1 -- start start 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f4a3f0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f4a3f0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4410b900 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f43fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f43fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41118/0 (socket says 172.21.15.37:41118) 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.182+0000 7f7f43fff700 1 -- 172.21.15.37:0/810214528 learned_addr learned my addr 172.21.15.37:0/810214528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f43fff700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4419ca30 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f43fff700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f38003f40 tx=0x7f7f38011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f380038b0 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f38003a50 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4419ccc0 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3801a5c0 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.183+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4419f9b0 con 0x7f7f441074d0 2024-04-25T13:16:39.408 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7f38024430 con 0x7f7f441074d0 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f417fa700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7f2c03c130 con 0x7f7f2c039560 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f3804daa0 con 0x7f7f441074d0 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f437fe700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.184+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.385+0000 7f7f437fe700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.385+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.786+0000 7f7f437fe700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:33.786+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:34.587+0000 7f7f437fe700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:34.587+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:36.189+0000 7f7f437fe700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2107975660 2024-04-25T13:16:39.409 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:36.189+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:37.835+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7f38024770 con 0x7f7f441074d0 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:37.835+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 msgr2=0x7f7f2c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:37.835+0000 7f7f417fa700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.836+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7f3804f420 con 0x7f7f441074d0 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.836+0000 7f7f417fa700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 0x7f7f2c03f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.837+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7f38007da0 con 0x7f7f2c03ce50 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.837+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 0x7f7f2c03f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.838+0000 7f7f437fe700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 0x7f7f2c03f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f34003a10 tx=0x7f7f34009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.839+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7f38007da0 con 0x7f7f2c03ce50 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7f4410ba80 con 0x7f7f2c03ce50 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f417fa700 1 -- 172.21.15.37:0/810214528 <== mgr.14120 v2:172.21.15.37:6800/3707854772 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7f4410ba80 con 0x7f7f2c03ce50 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 msgr2=0x7f7f2c03f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:39.410 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 0x7f7f2c03f240 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f34003a10 tx=0x7f7f34009290 comp rx=0 tx=0).stop 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 msgr2=0x7f7f4419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7f38003f40 tx=0x7f7f38011890 comp rx=0 tx=0).stop 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 shutdown_connections 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f7f2c03ce50 0x7f7f2c03f240 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f441074d0 0x7f7f4419c4f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.845+0000 7f7f4a3f0700 1 --2- 172.21.15.37:0/810214528 >> [v2:172.21.15.37:6800/2107975660,v1:172.21.15.37:6801/2107975660] conn(0x7f7f2c039560 0x7f7f2c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.846+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 >> 172.21.15.37:0/810214528 conn(0x7f7f44102ab0 msgr2=0x7f7f44103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.846+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 shutdown_connections 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:38.846+0000 7f7f4a3f0700 1 -- 172.21.15.37:0/810214528 wait complete. 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-04-25T13:16:39.411 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-04-25T13:16:40.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:39 smithi037 ceph-mon[29204]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:16:40.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:39 smithi037 ceph-mon[29204]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:16:40.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:39 smithi037 ceph-mon[29204]: [25/Apr/2024:13:16:39] ENGINE Bus STARTING 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f104ac700 1 Processor -- start 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f104ac700 1 -- start start 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f104ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f104ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08063cc0 con 0x7f0f081090e0 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f0e248700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f0e248700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39986/0 (socket says 172.21.15.37:39986) 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f0e248700 1 -- 172.21.15.37:0/1318889247 learned_addr learned my addr 172.21.15.37:0/1318889247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.801+0000 7f0f0e248700 1 -- 172.21.15.37:0/1318889247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0810ba10 con 0x7f0f081090e0 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f0e248700 1 --2- 172.21.15.37:0/1318889247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009e80 tx=0x7f0f0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590de9d9dcb188f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:40.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f0d246700 1 -- 172.21.15.37:0/1318889247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04017070 con 0x7f0f081090e0 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f0d246700 1 -- 172.21.15.37:0/1318889247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f0400b8c0 con 0x7f0f081090e0 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f0d246700 1 -- 172.21.15.37:0/1318889247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04004490 con 0x7f0f081090e0 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 -- 172.21.15.37:0/1318889247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 msgr2=0x7f0f0810b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 --2- 172.21.15.37:0/1318889247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009e80 tx=0x7f0f0400b330 comp rx=0 tx=0).stop 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 -- 172.21.15.37:0/1318889247 shutdown_connections 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 --2- 172.21.15.37:0/1318889247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0810b4d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 -- 172.21.15.37:0/1318889247 >> 172.21.15.37:0/1318889247 conn(0x7f0f08102ab0 msgr2=0x7f0f08104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 -- 172.21.15.37:0/1318889247 shutdown_connections 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.802+0000 7f0f104ac700 1 -- 172.21.15.37:0/1318889247 wait complete. 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f104ac700 1 Processor -- start 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f104ac700 1 -- start start 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f104ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f104ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0810c900 con 0x7f0f081090e0 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f0e248700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:40.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f0e248700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40002/0 (socket says 172.21.15.37:40002) 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.803+0000 7f0f0e248700 1 -- 172.21.15.37:0/535010590 learned_addr learned my addr 172.21.15.37:0/535010590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0f0e248700 1 -- 172.21.15.37:0/535010590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0819c980 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0f0e248700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b7b0 tx=0x7f0f0400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04020020 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0f04003d10 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0819cc10 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f04017040 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.804+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0819f900 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.805+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f0f04023020 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.805+0000 7f0eff7fe700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 0x7f0ef403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.805+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f080640a0 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.805+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f0400f050 con 0x7f0f081090e0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.805+0000 7f0f0da47700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 0x7f0ef403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.806+0000 7f0f0da47700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 0x7f0ef403b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006fd0 tx=0x7f0ef8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:40.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.810+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f0402a020 con 0x7f0f081090e0 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:39.891+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0f04012950 con 0x7f0f081090e0 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.036+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0f08103c60 con 0x7f0ef4039490 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.040+0000 7f0eff7fe700 1 -- 172.21.15.37:0/535010590 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f08103c60 con 0x7f0ef4039490 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 msgr2=0x7f0ef403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 0x7f0ef403b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006fd0 tx=0x7f0ef8006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 msgr2=0x7f0f0819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b7b0 tx=0x7f0f0400be90 comp rx=0 tx=0).stop 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 shutdown_connections 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f0ef4039490 0x7f0ef403b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 --2- 172.21.15.37:0/535010590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f081090e0 0x7f0f0819c440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 >> 172.21.15.37:0/535010590 conn(0x7f0f08102ab0 msgr2=0x7f0f08103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 shutdown_connections 2024-04-25T13:16:40.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:40.043+0000 7f0f104ac700 1 -- 172.21.15.37:0/535010590 wait complete. 2024-04-25T13:16:40.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: [25/Apr/2024:13:16:39] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: mgrmap e8: x(active, since 2s) 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: [25/Apr/2024:13:16:39] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: [25/Apr/2024:13:16:39] ENGINE Bus STARTED 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:40.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:40 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:41.874 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0fa55700 1 Processor -- start 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0fa55700 1 -- start start 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0fa55700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0fa55700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b08108ff0 con 0x7f3b08108600 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0d7f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0d7f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40012/0 (socket says 172.21.15.37:40012) 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.099+0000 7f3b0d7f1700 1 -- 172.21.15.37:0/3063105699 learned_addr learned my addr 172.21.15.37:0/3063105699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.100+0000 7f3b0d7f1700 1 -- 172.21.15.37:0/3063105699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b081097b0 con 0x7f3b08108600 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.100+0000 7f3b0d7f1700 1 --2- 172.21.15.37:0/3063105699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098a0 tx=0x7f3b04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db23b2a19cbdfa1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.100+0000 7f3afffff700 1 -- 172.21.15.37:0/3063105699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04004160 con 0x7f3b08108600 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.100+0000 7f3afffff700 1 -- 172.21.15.37:0/3063105699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b04004300 con 0x7f3b08108600 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.100+0000 7f3afffff700 1 -- 172.21.15.37:0/3063105699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04011af0 con 0x7f3b08108600 2024-04-25T13:16:41.875 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 -- 172.21.15.37:0/3063105699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 msgr2=0x7f3b08108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/3063105699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3b040098a0 tx=0x7f3b04009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 -- 172.21.15.37:0/3063105699 shutdown_connections 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/3063105699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b08108a20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 -- 172.21.15.37:0/3063105699 >> 172.21.15.37:0/3063105699 conn(0x7f3b08103bc0 msgr2=0x7f3b08105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 -- 172.21.15.37:0/3063105699 shutdown_connections 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.101+0000 7f3b0fa55700 1 -- 172.21.15.37:0/3063105699 wait complete. 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0fa55700 1 Processor -- start 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0fa55700 1 -- start start 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0fa55700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0fa55700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0810c9e0 con 0x7f3b08108600 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0d7f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0d7f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40016/0 (socket says 172.21.15.37:40016) 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0d7f1700 1 -- 172.21.15.37:0/2785848804 learned_addr learned my addr 172.21.15.37:0/2785848804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:41.876 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0d7f1700 1 -- 172.21.15.37:0/2785848804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0807e090 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.102+0000 7f3b0d7f1700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b04011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04003b20 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b0401a8c0 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0807c280 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b04024940 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0807c7c0 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.103+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3b04022020 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.104+0000 7f3afe7fc700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 0x7f3af403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.104+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b0404d7f0 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.104+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0810cb60 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.104+0000 7f3b0cff0700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 0x7f3af403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.104+0000 7f3b0cff0700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 0x7f3af403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.109+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b04021d80 con 0x7f3b08108600 2024-04-25T13:16:41.877 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.318+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3b0807ce70 con 0x7f3af40395b0 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.319+0000 7f3afe7fc700 1 -- 172.21.15.37:0/2785848804 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3b0807ce70 con 0x7f3af40395b0 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 msgr2=0x7f3af403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 0x7f3af403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 msgr2=0x7f3b0807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b04011b70 comp rx=0 tx=0).stop 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 shutdown_connections 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f3af40395b0 0x7f3af403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 --2- 172.21.15.37:0/2785848804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b08108600 0x7f3b0807db50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 >> 172.21.15.37:0/2785848804 conn(0x7f3b08103bc0 msgr2=0x7f3b081046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 shutdown_connections 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:41.321+0000 7f3b0fa55700 1 -- 172.21.15.37:0/2785848804 wait complete. 2024-04-25T13:16:41.878 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-04-25T13:16:42.219 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:41 smithi037 ceph-mon[29204]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:42.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: Generating public/private rsa key pair. 2024-04-25T13:16:42.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: Your identification has been saved in /tmp/tmpi0eveefg/key. 2024-04-25T13:16:42.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: Your public key has been saved in /tmp/tmpi0eveefg/key.pub. 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: The key fingerprint is: 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: SHA256:dASKwbJRdND1Z/Iutmn1cxew80JHWPL7b0vd6ANFjt8 ceph-d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: The key's randomart image is: 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: +---[RSA 3072]----+ 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | +=o..o.. | 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | o .+.. o ... | 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | +. . . + o+= | 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | . . . =.ooo | 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | S .o+..| 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | .o+.=E| 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | o.oo=.=| 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | ..+ o=.=| 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: | .o o*=| 2024-04-25T13:16:42.852 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: +----[SHA256]-----+ 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.326+0000 7fb27cf11700 1 Processor -- start 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.326+0000 7fb27cf11700 1 -- start start 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27cf11700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27cf11700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb27807c800 con 0x7fb27807de10 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40030/0 (socket says 172.21.15.37:40030) 2024-04-25T13:16:43.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27659c700 1 -- 172.21.15.37:0/1102195401 learned_addr learned my addr 172.21.15.37:0/1102195401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27659c700 1 -- 172.21.15.37:0/1102195401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27807c980 con 0x7fb27807de10 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27659c700 1 --2- 172.21.15.37:0/1102195401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098a0 tx=0x7fb260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f417b712b1a3c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27559a700 1 -- 172.21.15.37:0/1102195401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb260004160 con 0x7fb27807de10 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.327+0000 7fb27559a700 1 -- 172.21.15.37:0/1102195401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb260004300 con 0x7fb27807de10 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27559a700 1 -- 172.21.15.37:0/1102195401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb260011af0 con 0x7fb27807de10 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 -- 172.21.15.37:0/1102195401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 msgr2=0x7fb27807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1102195401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098a0 tx=0x7fb260009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 -- 172.21.15.37:0/1102195401 shutdown_connections 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1102195401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27807c2c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 -- 172.21.15.37:0/1102195401 >> 172.21.15.37:0/1102195401 conn(0x7fb278103b70 msgr2=0x7fb278105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 -- 172.21.15.37:0/1102195401 shutdown_connections 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.328+0000 7fb27cf11700 1 -- 172.21.15.37:0/1102195401 wait complete. 2024-04-25T13:16:43.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27cf11700 1 Processor -- start 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27cf11700 1 -- start start 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27cf11700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27cf11700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810ea60 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40044/0 (socket says 172.21.15.37:40044) 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27659c700 1 -- 172.21.15.37:0/1252580406 learned_addr learned my addr 172.21.15.37:0/1252580406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.329+0000 7fb27659c700 1 -- 172.21.15.37:0/1252580406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27819eb80 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb27659c700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb260011ac0 tx=0x7fb260011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb260003b20 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27819ee10 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb26001a8c0 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2600248d0 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.330+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a1b00 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.331+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb260022020 con 0x7fb27807de10 2024-04-25T13:16:43.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.331+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2780640a0 con 0x7fb27807de10 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.331+0000 7fb26f7fe700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 0x7fb26403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.331+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb26004d8f0 con 0x7fb27807de10 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.331+0000 7fb275d9b700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 0x7fb26403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.332+0000 7fb275d9b700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 0x7fb26403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb268006fd0 tx=0x7fb268006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.336+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb26002cdf0 con 0x7fb27807de10 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.543+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb278104df0 con 0x7fb2640395b0 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.730+0000 7fb26f7fe700 1 -- 172.21.15.37:0/1252580406 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb278104df0 con 0x7fb2640395b0 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 msgr2=0x7fb26403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 0x7fb26403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb268006fd0 tx=0x7fb268006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 msgr2=0x7fb27819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb260011ac0 tx=0x7fb260011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 shutdown_connections 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fb2640395b0 0x7fb26403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 --2- 172.21.15.37:0/1252580406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb27807de10 0x7fb27819e640 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:43.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 >> 172.21.15.37:0/1252580406 conn(0x7fb278103b70 msgr2=0x7fb2781046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:43.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 shutdown_connections 2024-04-25T13:16:43.293 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:42.733+0000 7fb27cf11700 1 -- 172.21.15.37:0/1252580406 wait complete. 2024-04-25T13:16:43.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:43 smithi037 ceph-mon[29204]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:43.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:43 smithi037 ceph-mon[29204]: Generating ssh key... 2024-04-25T13:16:43.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:43 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:43.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:43 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:44.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:16:44.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f509d2700 1 Processor -- start 2024-04-25T13:16:44.545 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f509d2700 1 -- start start 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f509d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f509d2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c07c8d0 con 0x7f6f4c07de50 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40056/0 (socket says 172.21.15.37:40056) 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.720+0000 7f6f4a59c700 1 -- 172.21.15.37:0/2298729652 learned_addr learned my addr 172.21.15.37:0/2298729652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.721+0000 7f6f4a59c700 1 -- 172.21.15.37:0/2298729652 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c07ca50 con 0x7f6f4c07de50 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.721+0000 7f6f4a59c700 1 --2- 172.21.15.37:0/2298729652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f340098a0 tx=0x7f6f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a22a7bd69db23cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.721+0000 7f6f4959a700 1 -- 172.21.15.37:0/2298729652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f34004160 con 0x7f6f4c07de50 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.721+0000 7f6f4959a700 1 -- 172.21.15.37:0/2298729652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f34004300 con 0x7f6f4c07de50 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.721+0000 7f6f4959a700 1 -- 172.21.15.37:0/2298729652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f34011af0 con 0x7f6f4c07de50 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 -- 172.21.15.37:0/2298729652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 msgr2=0x7f6f4c07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2298729652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6f340098a0 tx=0x7f6f34009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 -- 172.21.15.37:0/2298729652 shutdown_connections 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2298729652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c07c300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:44.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 -- 172.21.15.37:0/2298729652 >> 172.21.15.37:0/2298729652 conn(0x7f6f4c103bc0 msgr2=0x7f6f4c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 -- 172.21.15.37:0/2298729652 shutdown_connections 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 -- 172.21.15.37:0/2298729652 wait complete. 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.722+0000 7f6f509d2700 1 Processor -- start 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f509d2700 1 -- start start 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f509d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f509d2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10ead0 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f4a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40066/0 (socket says 172.21.15.37:40066) 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f4a59c700 1 -- 172.21.15.37:0/2253782700 learned_addr learned my addr 172.21.15.37:0/2253782700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f4a59c700 1 -- 172.21.15.37:0/2253782700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19eb80 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f4a59c700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011ac0 tx=0x7f6f34011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f34003b20 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f3401a8c0 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.723+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f340248d0 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.724+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19ee10 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.724+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a1b00 con 0x7f6f4c07de50 2024-04-25T13:16:44.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.724+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6f34022020 con 0x7f6f4c07de50 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.725+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c0640a0 con 0x7f6f4c07de50 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.725+0000 7f6f437fe700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 0x7f6f3803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.725+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6f3404d9f0 con 0x7f6f4c07de50 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.725+0000 7f6f49d9b700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 0x7f6f3803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.725+0000 7f6f49d9b700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 0x7f6f3803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006fd0 tx=0x7f6f3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.730+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f3401ade0 con 0x7f6f4c07de50 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.940+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6f4c104da0 con 0x7f6f380395b0 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.940+0000 7f6f437fe700 1 -- 172.21.15.37:0/2253782700 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6f4c104da0 con 0x7f6f380395b0 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.942+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 msgr2=0x7f6f3803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.942+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 0x7f6f3803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006fd0 tx=0x7f6f3c006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.942+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 msgr2=0x7f6f4c19e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.942+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011ac0 tx=0x7f6f34011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 shutdown_connections 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f6f380395b0 0x7f6f3803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:44.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 --2- 172.21.15.37:0/2253782700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f4c07de50 0x7f6f4c19e640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 >> 172.21.15.37:0/2253782700 conn(0x7f6f4c103bc0 msgr2=0x7f6f4c104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 shutdown_connections 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:43.943+0000 7f6f509d2700 1 -- 172.21.15.37:0/2253782700 wait complete. 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T13:16:44.549 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-04-25T13:16:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:44 smithi037 ceph-mon[29204]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:45 smithi037 ceph-mon[29204]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:47.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:46 smithi037 ceph-mon[29204]: Deploying cephadm binary to smithi037 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff31038700 1 Processor -- start 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff31038700 1 -- start start 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff31038700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff31038700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c108ff0 con 0x7eff2c108600 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40072/0 (socket says 172.21.15.37:40072) 2024-04-25T13:16:50.086 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.973+0000 7eff2ad9d700 1 -- 172.21.15.37:0/2596670179 learned_addr learned my addr 172.21.15.37:0/2596670179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.974+0000 7eff2ad9d700 1 -- 172.21.15.37:0/2596670179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1097b0 con 0x7eff2c108600 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.974+0000 7eff2ad9d700 1 --2- 172.21.15.37:0/2596670179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009e80 tx=0x7eff1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235b5ab277d236aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.974+0000 7eff29d9b700 1 -- 172.21.15.37:0/2596670179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff1c017070 con 0x7eff2c108600 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.974+0000 7eff29d9b700 1 -- 172.21.15.37:0/2596670179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff1c00b8c0 con 0x7eff2c108600 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.974+0000 7eff29d9b700 1 -- 172.21.15.37:0/2596670179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff1c004490 con 0x7eff2c108600 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 -- 172.21.15.37:0/2596670179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 msgr2=0x7eff2c108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 --2- 172.21.15.37:0/2596670179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009e80 tx=0x7eff1c00b330 comp rx=0 tx=0).stop 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 -- 172.21.15.37:0/2596670179 shutdown_connections 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 --2- 172.21.15.37:0/2596670179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c108a20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 -- 172.21.15.37:0/2596670179 >> 172.21.15.37:0/2596670179 conn(0x7eff2c103bc0 msgr2=0x7eff2c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 -- 172.21.15.37:0/2596670179 shutdown_connections 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.975+0000 7eff31038700 1 -- 172.21.15.37:0/2596670179 wait complete. 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff31038700 1 Processor -- start 2024-04-25T13:16:50.087 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff31038700 1 -- start start 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff31038700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff31038700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10c9e0 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff2ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40088/0 (socket says 172.21.15.37:40088) 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.976+0000 7eff2ad9d700 1 -- 172.21.15.37:0/1559758091 learned_addr learned my addr 172.21.15.37:0/1559758091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff2ad9d700 1 -- 172.21.15.37:0/1559758091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c19c970 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff2ad9d700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7eff1c004a60 tx=0x7eff1c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff1c020020 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff1c003c60 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c19cc00 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff1c017040 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.977+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c19f8f0 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.978+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7eff1c023020 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.978+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c10cb60 con 0x7eff2c108600 2024-04-25T13:16:50.088 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.978+0000 7eff23fff700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 0x7eff1803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.978+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7eff1c00f050 con 0x7eff2c108600 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.978+0000 7eff2a59c700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 0x7eff1803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.979+0000 7eff2a59c700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 0x7eff1803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff14006fd0 tx=0x7eff14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:44.983+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff1c02a020 con 0x7eff2c108600 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:45.200+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7eff2c104d90 con 0x7eff18039600 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.480+0000 7eff23fff700 1 -- 172.21.15.37:0/1559758091 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7eff2c104d90 con 0x7eff18039600 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 msgr2=0x7eff1803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 0x7eff1803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff14006fd0 tx=0x7eff14006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 msgr2=0x7eff2c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7eff1c004a60 tx=0x7eff1c00bd50 comp rx=0 tx=0).stop 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 shutdown_connections 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7eff18039600 0x7eff1803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:50.089 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 --2- 172.21.15.37:0/1559758091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff2c108600 0x7eff2c19c430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:50.090 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 >> 172.21.15.37:0/1559758091 conn(0x7eff2c103bc0 msgr2=0x7eff2c104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:50.090 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 shutdown_connections 2024-04-25T13:16:50.090 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:49.483+0000 7eff31038700 1 -- 172.21.15.37:0/1559758091 wait complete. 2024-04-25T13:16:50.090 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-04-25T13:16:50.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:50 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:50.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:50 smithi037 ceph-mon[29204]: Added host smithi037 2024-04-25T13:16:50.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:50 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:16:51.552 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.569+0000 7fccb3e5b700 1 Processor -- start 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.569+0000 7fccb3e5b700 1 -- start start 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb3e5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb3e5b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccac108fb0 con 0x7fccac1085c0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb1bf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb1bf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45118/0 (socket says 172.21.15.37:45118) 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb1bf7700 1 -- 172.21.15.37:0/491068729 learned_addr learned my addr 172.21.15.37:0/491068729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb1bf7700 1 -- 172.21.15.37:0/491068729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac1097c0 con 0x7fccac1085c0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb1bf7700 1 --2- 172.21.15.37:0/491068729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcca80098a0 tx=0x7fcca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f06489ceefb60a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb0bf5700 1 -- 172.21.15.37:0/491068729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcca8004160 con 0x7fccac1085c0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb0bf5700 1 -- 172.21.15.37:0/491068729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcca8004300 con 0x7fccac1085c0 2024-04-25T13:16:51.553 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.570+0000 7fccb0bf5700 1 -- 172.21.15.37:0/491068729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcca8011af0 con 0x7fccac1085c0 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 -- 172.21.15.37:0/491068729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 msgr2=0x7fccac1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/491068729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcca80098a0 tx=0x7fcca8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 -- 172.21.15.37:0/491068729 shutdown_connections 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/491068729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac1089e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 -- 172.21.15.37:0/491068729 >> 172.21.15.37:0/491068729 conn(0x7fccac103be0 msgr2=0x7fccac106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 -- 172.21.15.37:0/491068729 shutdown_connections 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.571+0000 7fccb3e5b700 1 -- 172.21.15.37:0/491068729 wait complete. 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb3e5b700 1 Processor -- start 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb3e5b700 1 -- start start 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb3e5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb3e5b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccac10c9f0 con 0x7fccac1085c0 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb1bf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb1bf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45132/0 (socket says 172.21.15.37:45132) 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb1bf7700 1 -- 172.21.15.37:0/2001952483 learned_addr learned my addr 172.21.15.37:0/2001952483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:51.554 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.572+0000 7fccb1bf7700 1 -- 172.21.15.37:0/2001952483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac19ca30 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fccb1bf7700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcca8003f40 tx=0x7fcca8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcca80038b0 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcca8003a50 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac19ccc0 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcca801a5c0 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.573+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac19f9b0 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcca801a870 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac10cb70 con 0x7fccac1085c0 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fcca2ffd700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 0x7fcc9803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:51.555 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcca804da90 con 0x7fccac1085c0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fccb13f6700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 0x7fcc9803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.574+0000 7fccb13f6700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 0x7fcc9803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.579+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcca8019c70 con 0x7fccac1085c0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.815+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fccac104e80 con 0x7fcc980395b0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.822+0000 7fcca2ffd700 1 -- 172.21.15.37:0/2001952483 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fccac104e80 con 0x7fcc980395b0 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 msgr2=0x7fcc9803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 0x7fcc9803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c006fd0 tx=0x7fcc9c006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 msgr2=0x7fccac19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcca8003f40 tx=0x7fcca8011890 comp rx=0 tx=0).stop 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 shutdown_connections 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fcc980395b0 0x7fcc9803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 --2- 172.21.15.37:0/2001952483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccac1085c0 0x7fccac19c4f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 >> 172.21.15.37:0/2001952483 conn(0x7fccac103be0 msgr2=0x7fccac104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:51.556 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 shutdown_connections 2024-04-25T13:16:51.557 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:50.825+0000 7fccb3e5b700 1 -- 172.21.15.37:0/2001952483 wait complete. 2024-04-25T13:16:51.557 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-04-25T13:16:52.074 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:51 smithi037 ceph-mon[29204]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:52.074 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:51 smithi037 ceph-mon[29204]: Saving service mon spec with placement count:5 2024-04-25T13:16:52.074 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:51 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:53.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:53 smithi037 ceph-mon[29204]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:16:53.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:53 smithi037 ceph-mon[29204]: Saving service mgr spec with placement count:2 2024-04-25T13:16:53.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:53 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:53.624 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:53 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7c379700 1 Processor -- start 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7c379700 1 -- start start 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7c379700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7c379700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7407be10 con 0x7f9e74106ed0 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7a115700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7a115700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45140/0 (socket says 172.21.15.37:45140) 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7a115700 1 -- 172.21.15.37:0/2436873739 learned_addr learned my addr 172.21.15.37:0/2436873739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.182+0000 7f9e7a115700 1 -- 172.21.15.37:0/2436873739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e74107830 con 0x7f9e74106ed0 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7a115700 1 --2- 172.21.15.37:0/2436873739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e700098a0 tx=0x7f9e70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fac1d6a1acf16f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:53.631 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e79113700 1 -- 172.21.15.37:0/2436873739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70004160 con 0x7f9e74106ed0 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e79113700 1 -- 172.21.15.37:0/2436873739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9e70004300 con 0x7f9e74106ed0 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e79113700 1 -- 172.21.15.37:0/2436873739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70011af0 con 0x7f9e74106ed0 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 -- 172.21.15.37:0/2436873739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 msgr2=0x7f9e741072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2436873739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e700098a0 tx=0x7f9e70009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 -- 172.21.15.37:0/2436873739 shutdown_connections 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2436873739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e741072f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 -- 172.21.15.37:0/2436873739 >> 172.21.15.37:0/2436873739 conn(0x7f9e74102a90 msgr2=0x7f9e74104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 -- 172.21.15.37:0/2436873739 shutdown_connections 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.183+0000 7f9e7c379700 1 -- 172.21.15.37:0/2436873739 wait complete. 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7c379700 1 Processor -- start 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7c379700 1 -- start start 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7c379700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7c379700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741087f0 con 0x7f9e74106ed0 2024-04-25T13:16:53.632 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7a115700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7a115700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45154/0 (socket says 172.21.15.37:45154) 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.184+0000 7f9e7a115700 1 -- 172.21.15.37:0/2565168504 learned_addr learned my addr 172.21.15.37:0/2565168504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e7a115700 1 -- 172.21.15.37:0/2565168504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7419ebc0 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e7a115700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e70011ac0 tx=0x7f9e70011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e70003b20 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9e7001a8c0 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e700248d0 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7419ee50 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.185+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741a1b40 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.186+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9e70022020 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.186+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e740640a0 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.186+0000 7f9e6b7fe700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 0x7f9e6003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.186+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9e7004d800 con 0x7f9e74106ed0 2024-04-25T13:16:53.633 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.186+0000 7f9e79914700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 0x7f9e6003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.187+0000 7f9e79914700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 0x7f9e6003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.191+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e7000f2e0 con 0x7f9e74106ed0 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.415+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9e7402e260 con 0x7f9e60039600 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.420+0000 7f9e6b7fe700 1 -- 172.21.15.37:0/2565168504 <== mgr.14120 v2:172.21.15.37:6800/3707854772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9e7402e260 con 0x7f9e60039600 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 msgr2=0x7f9e6003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 0x7f9e6003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006fd0 tx=0x7f9e64006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 msgr2=0x7f9e7419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9e70011ac0 tx=0x7f9e70011ba0 comp rx=0 tx=0).stop 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 shutdown_connections 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f9e60039600 0x7f9e6003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 --2- 172.21.15.37:0/2565168504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e74106ed0 0x7f9e7419e680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 >> 172.21.15.37:0/2565168504 conn(0x7f9e74102a90 msgr2=0x7f9e741952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 shutdown_connections 2024-04-25T13:16:53.634 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:52.422+0000 7f9e7c379700 1 -- 172.21.15.37:0/2565168504 wait complete. 2024-04-25T13:16:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:54 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2016034209' entity='client.admin' 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc72763700 1 Processor -- start 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc72763700 1 -- start start 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc72763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc72763700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c108fa0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45160/0 (socket says 172.21.15.37:45160) 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc6bfff700 1 -- 172.21.15.37:0/1309564760 learned_addr learned my addr 172.21.15.37:0/1309564760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.101+0000 7fdc6bfff700 1 -- 172.21.15.37:0/1309564760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c109760 con 0x7fdc6c1085b0 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.102+0000 7fdc6bfff700 1 --2- 172.21.15.37:0/1309564760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0098a0 tx=0x7fdc5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba20d83a9c43b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:55.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.102+0000 7fdc6affd700 1 -- 172.21.15.37:0/1309564760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c004160 con 0x7fdc6c1085b0 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.102+0000 7fdc6affd700 1 -- 172.21.15.37:0/1309564760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c004300 con 0x7fdc6c1085b0 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.102+0000 7fdc6affd700 1 -- 172.21.15.37:0/1309564760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c011a20 con 0x7fdc6c1085b0 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- 172.21.15.37:0/1309564760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 msgr2=0x7fdc6c1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 --2- 172.21.15.37:0/1309564760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0098a0 tx=0x7fdc5c009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- 172.21.15.37:0/1309564760 shutdown_connections 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 --2- 172.21.15.37:0/1309564760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c1089d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- 172.21.15.37:0/1309564760 >> 172.21.15.37:0/1309564760 conn(0x7fdc6c103b70 msgr2=0x7fdc6c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- 172.21.15.37:0/1309564760 shutdown_connections 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- 172.21.15.37:0/1309564760 wait complete. 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 Processor -- start 2024-04-25T13:16:55.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.103+0000 7fdc72763700 1 -- start start 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc72763700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc72763700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c10cb10 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc6bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45162/0 (socket says 172.21.15.37:45162) 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc6bfff700 1 -- 172.21.15.37:0/2016034209 learned_addr learned my addr 172.21.15.37:0/2016034209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc6bfff700 1 -- 172.21.15.37:0/2016034209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19c970 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.104+0000 7fdc6bfff700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0116e0 tx=0x7fdc5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.105+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c0039e0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.105+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c01b8f0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.105+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19cc00 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.105+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc5c023760 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.105+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c19f8f0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdc5c0239a0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c10cc90 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc697fa700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 0x7fdc5403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc5c04e010 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc6b7fe700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 0x7fdc5403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.106+0000 7fdc6b7fe700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 0x7fdc5403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.111+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc5c01f030 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.334+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdc6c19fac0 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.336+0000 7fdc697fa700 1 -- 172.21.15.37:0/2016034209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdc5c01bd50 con 0x7fdc6c1085b0 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 msgr2=0x7fdc5403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 0x7fdc5403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc60006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 msgr2=0x7fdc6c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0116e0 tx=0x7fdc5c011aa0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 shutdown_connections 2024-04-25T13:16:55.585 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fdc540395b0 0x7fdc5403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.586 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 --2- 172.21.15.37:0/2016034209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc6c1085b0 0x7fdc6c19c430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:55.586 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 >> 172.21.15.37:0/2016034209 conn(0x7fdc6c103b70 msgr2=0x7fdc6c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:55.586 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 shutdown_connections 2024-04-25T13:16:55.586 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:54.340+0000 7fdc72763700 1 -- 172.21.15.37:0/2016034209 wait complete. 2024-04-25T13:16:56.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:56 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:56.938 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.213+0000 7fa9e4714700 1 Processor -- start 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.213+0000 7fa9e4714700 1 -- start start 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.213+0000 7fa9e4714700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.213+0000 7fa9e4714700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc107290 con 0x7fa9dc1068a0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.214+0000 7fa9e24b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.214+0000 7fa9e24b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45172/0 (socket says 172.21.15.37:45172) 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.214+0000 7fa9e24b0700 1 -- 172.21.15.37:0/2886000124 learned_addr learned my addr 172.21.15.37:0/2886000124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.214+0000 7fa9e24b0700 1 -- 172.21.15.37:0/2886000124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc109830 con 0x7fa9dc1068a0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.214+0000 7fa9e24b0700 1 --2- 172.21.15.37:0/2886000124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80098a0 tx=0x7fa9d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1619b7378fc8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e14ae700 1 -- 172.21.15.37:0/2886000124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8004160 con 0x7fa9dc1068a0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e14ae700 1 -- 172.21.15.37:0/2886000124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d8004300 con 0x7fa9dc1068a0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e14ae700 1 -- 172.21.15.37:0/2886000124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8011a20 con 0x7fa9dc1068a0 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e4714700 1 -- 172.21.15.37:0/2886000124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 msgr2=0x7fa9dc106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2886000124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80098a0 tx=0x7fa9d8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e4714700 1 -- 172.21.15.37:0/2886000124 shutdown_connections 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2886000124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc106cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.215+0000 7fa9e4714700 1 -- 172.21.15.37:0/2886000124 >> 172.21.15.37:0/2886000124 conn(0x7fa9dc1021d0 msgr2=0x7fa9dc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 -- 172.21.15.37:0/2886000124 shutdown_connections 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 -- 172.21.15.37:0/2886000124 wait complete. 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 Processor -- start 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 -- start start 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.216+0000 7fa9e4714700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc10cbe0 con 0x7fa9dc1068a0 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e24b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e24b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45180/0 (socket says 172.21.15.37:45180) 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e24b0700 1 -- 172.21.15.37:0/2086175870 learned_addr learned my addr 172.21.15.37:0/2086175870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e24b0700 1 -- 172.21.15.37:0/2086175870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc19ec80 con 0x7fa9dc1068a0 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e24b0700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000c00 tx=0x7fa9d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:56.940 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8003aa0 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d801b8f0 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc19ef10 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8023890 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.217+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1a1c00 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.218+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc10cd60 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.219+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa9d801b440 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.219+0000 7fa9d37fe700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 0x7fa9c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.219+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa9d804de80 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.219+0000 7fa9e1caf700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 0x7fa9c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.220+0000 7fa9e1caf700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 0x7fa9c803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006fd0 tx=0x7fa9cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.224+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9d801f030 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.432+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa9dc1a21c0 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.436+0000 7fa9d37fe700 1 -- 172.21.15.37:0/2086175870 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa9d80292a0 con 0x7fa9dc1068a0 2024-04-25T13:16:56.941 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 msgr2=0x7fa9c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 0x7fa9c803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006fd0 tx=0x7fa9cc006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 msgr2=0x7fa9dc19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8000c00 tx=0x7fa9d8011aa0 comp rx=0 tx=0).stop 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 shutdown_connections 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fa9c80395b0 0x7fa9c803ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 --2- 172.21.15.37:0/2086175870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9dc1068a0 0x7fa9dc19e740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.438+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 >> 172.21.15.37:0/2086175870 conn(0x7fa9dc1021d0 msgr2=0x7fa9dc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.439+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 shutdown_connections 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:56.439+0000 7fa9e4714700 1 -- 172.21.15.37:0/2086175870 wait complete. 2024-04-25T13:16:56.942 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-04-25T13:16:57.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:57 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2086175870' entity='client.admin' 2024-04-25T13:16:57.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:57 smithi037 ceph-mon[29204]: from='mgr.14120 172.21.15.37:0/3605052288' entity='mgr.x' 2024-04-25T13:16:58.647 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1232538537' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T13:16:58.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setuser ceph since I am not root 2024-04-25T13:16:58.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setgroup ceph since I am not root 2024-04-25T13:16:58.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:58.626+0000 7fee71201200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:16:58.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:58.819+0000 7fee71201200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:16:58.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:58.927+0000 7fee71201200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:16:59.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.405+0000 7f5e33a7a700 1 Processor -- start 2024-04-25T13:16:59.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e33a7a700 1 -- start start 2024-04-25T13:16:59.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e33a7a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:59.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e33a7a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c063d20 con 0x7f5e2c109080 2024-04-25T13:16:59.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e31816700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e31816700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45184/0 (socket says 172.21.15.37:45184) 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e31816700 1 -- 172.21.15.37:0/3613512286 learned_addr learned my addr 172.21.15.37:0/3613512286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e31816700 1 -- 172.21.15.37:0/3613512286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c10b9b0 con 0x7f5e2c109080 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.406+0000 7f5e31816700 1 --2- 172.21.15.37:0/3613512286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e280098a0 tx=0x7f5e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2c373ed910a998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.407+0000 7f5e30814700 1 -- 172.21.15.37:0/3613512286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e28004160 con 0x7f5e2c109080 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.407+0000 7f5e30814700 1 -- 172.21.15.37:0/3613512286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e28004300 con 0x7f5e2c109080 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.407+0000 7f5e30814700 1 -- 172.21.15.37:0/3613512286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e28011a20 con 0x7f5e2c109080 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.407+0000 7f5e33a7a700 1 -- 172.21.15.37:0/3613512286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 msgr2=0x7f5e2c10b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/3613512286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e280098a0 tx=0x7f5e28009bb0 comp rx=0 tx=0).stop 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 -- 172.21.15.37:0/3613512286 shutdown_connections 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/3613512286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c10b470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 -- 172.21.15.37:0/3613512286 >> 172.21.15.37:0/3613512286 conn(0x7f5e2c102ab0 msgr2=0x7f5e2c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 -- 172.21.15.37:0/3613512286 shutdown_connections 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 -- 172.21.15.37:0/3613512286 wait complete. 2024-04-25T13:16:59.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.408+0000 7f5e33a7a700 1 Processor -- start 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e33a7a700 1 -- start start 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e33a7a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e33a7a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c10c9b0 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e31816700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e31816700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45190/0 (socket says 172.21.15.37:45190) 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e31816700 1 -- 172.21.15.37:0/1232538537 learned_addr learned my addr 172.21.15.37:0/1232538537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e31816700 1 -- 172.21.15.37:0/1232538537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c19c940 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.409+0000 7f5e31816700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e280116e0 tx=0x7f5e28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e28003a60 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e2801b8f0 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c19cbd0 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e28019860 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c19f8c0 con 0x7f5e2c109080 2024-04-25T13:16:59.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.410+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5e28019b60 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.411+0000 7f5e22ffd700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 0x7f5e1803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.411+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e2804ec80 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.411+0000 7f5e31015700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 0x7f5e1803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.411+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e10005360 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.411+0000 7f5e31015700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 0x7f5e1803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c006fd0 tx=0x7f5e1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.416+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e2801f030 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:57.693+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5e10005650 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.440+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5e28029260 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.442+0000 7f5e22ffd700 1 -- 172.21.15.37:0/1232538537 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5e2801a8f0 con 0x7f5e2c109080 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.443+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 msgr2=0x7f5e1803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.443+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 0x7f5e1803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c006fd0 tx=0x7f5e1c006e40 comp rx=0 tx=0).stop 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 msgr2=0x7f5e2c19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e280116e0 tx=0x7f5e28011aa0 comp rx=0 tx=0).stop 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 shutdown_connections 2024-04-25T13:16:59.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f5e18039560 0x7f5e1803ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:59.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 --2- 172.21.15.37:0/1232538537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e2c109080 0x7f5e2c19c400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:16:59.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 >> 172.21.15.37:0/1232538537 conn(0x7f5e2c102ab0 msgr2=0x7f5e2c103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:16:59.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 shutdown_connections 2024-04-25T13:16:59.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:58.444+0000 7f5e33a7a700 1 -- 172.21.15.37:0/1232538537 wait complete. 2024-04-25T13:16:59.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:59 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1232538537' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T13:16:59.562 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:16:59 smithi037 ceph-mon[29204]: mgrmap e9: x(active, since 20s) 2024-04-25T13:16:59.562 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:59 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:59.255+0000 7fee71201200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:16:59.562 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:16:59 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:16:59.345+0000 7fee71201200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:17:00.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3068226823' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T13:17:00.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:17:00.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.666+0000 7fe834c1a700 1 Processor -- start 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.666+0000 7fe834c1a700 1 -- start start 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe834c1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe834c1a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe830063d20 con 0x7fe830106e70 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54012/0 (socket says 172.21.15.37:54012) 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82e59c700 1 -- 172.21.15.37:0/212492770 learned_addr learned my addr 172.21.15.37:0/212492770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82e59c700 1 -- 172.21.15.37:0/212492770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8301077d0 con 0x7fe830106e70 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82e59c700 1 --2- 172.21.15.37:0/212492770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098a0 tx=0x7fe820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2b49546b0d3450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82d59a700 1 -- 172.21.15.37:0/212492770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820004160 con 0x7fe830106e70 2024-04-25T13:17:00.666 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82d59a700 1 -- 172.21.15.37:0/212492770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe820004300 con 0x7fe830106e70 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.667+0000 7fe82d59a700 1 -- 172.21.15.37:0/212492770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820011a20 con 0x7fe830106e70 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 -- 172.21.15.37:0/212492770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 msgr2=0x7fe830107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 --2- 172.21.15.37:0/212492770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098a0 tx=0x7fe820009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 -- 172.21.15.37:0/212492770 shutdown_connections 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 --2- 172.21.15.37:0/212492770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe830107290 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 -- 172.21.15.37:0/212492770 >> 172.21.15.37:0/212492770 conn(0x7fe830102ab0 msgr2=0x7fe830104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 -- 172.21.15.37:0/212492770 shutdown_connections 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.668+0000 7fe834c1a700 1 -- 172.21.15.37:0/212492770 wait complete. 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe834c1a700 1 Processor -- start 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe834c1a700 1 -- start start 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe834c1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe834c1a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe830108840 con 0x7fe830106e70 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe82e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:00.667 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe82e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54018/0 (socket says 172.21.15.37:54018) 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.669+0000 7fe82e59c700 1 -- 172.21.15.37:0/3068226823 learned_addr learned my addr 172.21.15.37:0/3068226823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe82e59c700 1 -- 172.21.15.37:0/3068226823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83019eb60 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe82e59c700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8200116e0 tx=0x7fe820011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820003a60 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82001b8f0 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83019edf0 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe820019860 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.670+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a1ae0 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe820019b60 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8300640a0 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe81f7fe700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 0x7fe81803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe82004dd50 con 0x7fe830106e70 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe82dd9b700 1 -- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 msgr2=0x7fe81803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.671+0000 7fe82dd9b700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 0x7fe81803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:17:00.668 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.676+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe820051030 con 0x7fe830106e70 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.872+0000 7fe82dd9b700 1 -- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 msgr2=0x7fe81803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.872+0000 7fe82dd9b700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 0x7fe81803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.989+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe83002e110 con 0x7fe830106e70 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.989+0000 7fe81f7fe700 1 -- 172.21.15.37:0/3068226823 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe82001f030 con 0x7fe830106e70 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 msgr2=0x7fe81803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 0x7fe81803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 msgr2=0x7fe83019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8200116e0 tx=0x7fe820011aa0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 shutdown_connections 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7fe8180395b0 0x7fe81803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 --2- 172.21.15.37:0/3068226823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe830106e70 0x7fe83019e620 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.991+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 >> 172.21.15.37:0/3068226823 conn(0x7fe830102ab0 msgr2=0x7fe830195290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.992+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 shutdown_connections 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:16:59.992+0000 7fe834c1a700 1 -- 172.21.15.37:0/3068226823 wait complete. 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-25T13:17:00.669 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-04-25T13:17:00.916 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:00.639+0000 7fee71201200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:17:00.916 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:00.822+0000 7fee71201200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:17:01.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:01.000+0000 7fee71201200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:17:01.307 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:01.306+0000 7fee71201200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:17:01.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:01.404+0000 7fee71201200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:17:02.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:01.615+0000 7fee71201200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:17:02.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:01.817+0000 7fee71201200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:17:02.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:02.445+0000 7fee71201200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:17:03.196 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:02.886+0000 7fee71201200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:17:03.196 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:02.986+0000 7fee71201200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:17:03.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:03.196+0000 7fee71201200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:17:04.094 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:03.742+0000 7fee71201200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:17:04.094 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:03.839+0000 7fee71201200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:17:04.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:04.093+0000 7fee71201200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:17:04.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:04.233+0000 7fee71201200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:17:04.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:04.332+0000 7fee71201200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:17:04.782 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:04.454+0000 7fee71201200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:17:04.782 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:04.781+0000 7fee71201200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:17:05.696 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:05.413+0000 7fee71201200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:17:05.696 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:17:05.640+0000 7fee71201200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: Active manager daemon x restarted 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: Activating manager daemon x 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: mgrmap e10: x(active, starting, since 0.00335781s) 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:17:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: Manager daemon x is now available 2024-04-25T13:17:06.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:07.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:17:07.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:17:07.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:06 smithi037 ceph-mon[29204]: [25/Apr/2024:13:17:06] ENGINE Bus STARTING 2024-04-25T13:17:07.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:06 smithi037 ceph-mon[29204]: mgrmap e11: x(active, since 1.0058s) 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0803477700 1 Processor -- start 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0803477700 1 -- start start 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0803477700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0803477700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10a5a0 con 0x7f07fc109bb0 2024-04-25T13:17:07.652 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0801213700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0801213700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54034/0 (socket says 172.21.15.37:54034) 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.129+0000 7f0801213700 1 -- 172.21.15.37:0/4288673674 learned_addr learned my addr 172.21.15.37:0/4288673674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.130+0000 7f0801213700 1 -- 172.21.15.37:0/4288673674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc10a720 con 0x7f07fc109bb0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.130+0000 7f0801213700 1 --2- 172.21.15.37:0/4288673674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098a0 tx=0x7f07f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a2f6ce3b691524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.130+0000 7f07f3fff700 1 -- 172.21.15.37:0/4288673674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f8004160 con 0x7f07fc109bb0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.130+0000 7f07f3fff700 1 -- 172.21.15.37:0/4288673674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f8004300 con 0x7f07fc109bb0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.130+0000 7f07f3fff700 1 -- 172.21.15.37:0/4288673674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f8011a20 con 0x7f07fc109bb0 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- 172.21.15.37:0/4288673674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 msgr2=0x7f07fc109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 --2- 172.21.15.37:0/4288673674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098a0 tx=0x7f07f8009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- 172.21.15.37:0/4288673674 shutdown_connections 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 --2- 172.21.15.37:0/4288673674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc109fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- 172.21.15.37:0/4288673674 >> 172.21.15.37:0/4288673674 conn(0x7f07fc1021d0 msgr2=0x7f07fc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- 172.21.15.37:0/4288673674 shutdown_connections 2024-04-25T13:17:07.653 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- 172.21.15.37:0/4288673674 wait complete. 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 Processor -- start 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.131+0000 7f0803477700 1 -- start start 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0803477700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0803477700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc107460 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0801213700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0801213700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54046/0 (socket says 172.21.15.37:54046) 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0801213700 1 -- 172.21.15.37:0/1928541382 learned_addr learned my addr 172.21.15.37:0/1928541382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0801213700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc109710 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0801213700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07f80119f0 tx=0x7f07f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f80039e0 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f801b8f0 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc107900 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07f80237a0 con 0x7f07fc109bb0 2024-04-25T13:17:07.654 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.132+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc107e40 con 0x7f07fc109bb0 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.133+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f07f801b440 con 0x7f07fc109bb0 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.133+0000 7f07f27fc700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.133+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07e803c180 con 0x7f07e80395b0 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.134+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07f804df30 con 0x7f07fc109bb0 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.134+0000 7f0800a12700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.134+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.335+0000 7f0800a12700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.335+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.736+0000 7f0800a12700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:01.736+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:02.537+0000 7f0800a12700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:02.537+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:04.138+0000 7f0800a12700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3707854772 2024-04-25T13:17:07.655 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:04.138+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:05.644+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f07f802c910 con 0x7f07fc109bb0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:05.644+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 msgr2=0x7f07e803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:05.644+0000 7f07f27fc700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.645+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f07f804fac0 con 0x7f07fc109bb0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.646+0000 7f07f27fc700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 0x7f07e803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.646+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f07f80068a0 con 0x7f07e803cfe0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.646+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 0x7f07e803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.647+0000 7f0800a12700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 0x7f07e803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07ec003a10 tx=0x7f07ec0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.647+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f07f80068a0 con 0x7f07e803cfe0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.654+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f07fc069b80 con 0x7f07e803cfe0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f07f27fc700 1 -- 172.21.15.37:0/1928541382 <== mgr.14152 v2:172.21.15.37:6800/2819711067 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f07fc069b80 con 0x7f07e803cfe0 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 msgr2=0x7f07e803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.656 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 0x7f07e803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f07ec003a10 tx=0x7f07ec0092b0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 msgr2=0x7f07fc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f07f80119f0 tx=0x7f07f8011ad0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 shutdown_connections 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07e803cfe0 0x7f07e803f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07fc109bb0 0x7f07fc1091d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 --2- 172.21.15.37:0/1928541382 >> [v2:172.21.15.37:6800/3707854772,v1:172.21.15.37:6801/3707854772] conn(0x7f07e80395b0 0x7f07e803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 >> 172.21.15.37:0/1928541382 conn(0x7f07fc1021d0 msgr2=0x7f07fc102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 shutdown_connections 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:06.655+0000 7f0803477700 1 -- 172.21.15.37:0/1928541382 wait complete. 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-04-25T13:17:07.657 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-04-25T13:17:07.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:07 smithi037 ceph-mon[29204]: [25/Apr/2024:13:17:06] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T13:17:07.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:07 smithi037 ceph-mon[29204]: [25/Apr/2024:13:17:06] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T13:17:07.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:07 smithi037 ceph-mon[29204]: [25/Apr/2024:13:17:06] ENGINE Bus STARTED 2024-04-25T13:17:07.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:07 smithi037 ceph-mon[29204]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T13:17:07.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:07 smithi037 ceph-mon[29204]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T13:17:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:08 smithi037 ceph-mon[29204]: mgrmap e12: x(active, since 2s) 2024-04-25T13:17:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:08 smithi037 ceph-mon[29204]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:09.132 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f7745930700 1 Processor -- start 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f7745930700 1 -- start start 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f7745930700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f7745930700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7740107ee0 con 0x7f77401074f0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f773effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f773effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54130/0 (socket says 172.21.15.37:54130) 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f773effd700 1 -- 172.21.15.37:0/2608219856 learned_addr learned my addr 172.21.15.37:0/2608219856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.131+0000 7f773effd700 1 -- 172.21.15.37:0/2608219856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77401086f0 con 0x7f77401074f0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.132+0000 7f773effd700 1 --2- 172.21.15.37:0/2608219856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77300098a0 tx=0x7f7730009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1bbc309d528a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.132+0000 7f773dffb700 1 -- 172.21.15.37:0/2608219856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7730004160 con 0x7f77401074f0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.132+0000 7f773dffb700 1 -- 172.21.15.37:0/2608219856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7730004300 con 0x7f77401074f0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.132+0000 7f773dffb700 1 -- 172.21.15.37:0/2608219856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7730011a20 con 0x7f77401074f0 2024-04-25T13:17:09.133 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- 172.21.15.37:0/2608219856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 msgr2=0x7f7740107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 --2- 172.21.15.37:0/2608219856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77300098a0 tx=0x7f7730009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- 172.21.15.37:0/2608219856 shutdown_connections 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 --2- 172.21.15.37:0/2608219856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f7740107910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- 172.21.15.37:0/2608219856 >> 172.21.15.37:0/2608219856 conn(0x7f7740102a90 msgr2=0x7f7740104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- 172.21.15.37:0/2608219856 shutdown_connections 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- 172.21.15.37:0/2608219856 wait complete. 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 Processor -- start 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.133+0000 7f7745930700 1 -- start start 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f7745930700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f7745930700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f774010baa0 con 0x7f77401074f0 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f773effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f773effd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54132/0 (socket says 172.21.15.37:54132) 2024-04-25T13:17:09.134 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f773effd700 1 -- 172.21.15.37:0/2297568409 learned_addr learned my addr 172.21.15.37:0/2297568409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f773effd700 1 -- 172.21.15.37:0/2297568409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774019c9c0 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f773effd700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7730009520 tx=0x7f7730011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77300039e0 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f773001b8f0 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7730019770 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.134+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774019cc50 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.135+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774019f940 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f773001b440 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f774492e700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 0x7f772c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f773004dfc0 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774010bc20 con 0x7f77401074f0 2024-04-25T13:17:09.135 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f773e7fc700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 0x7f772c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.136+0000 7f773e7fc700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 0x7f772c03bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7728006fd0 tx=0x7f7728006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.146+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f773001f030 con 0x7f77401074f0 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.371+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f774002e100 con 0x7f772c039650 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.514+0000 7f774492e700 1 -- 172.21.15.37:0/2297568409 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f774002e100 con 0x7f772c039650 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 msgr2=0x7f772c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 0x7f772c03bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7728006fd0 tx=0x7f7728006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 msgr2=0x7f774019c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7730009520 tx=0x7f7730011aa0 comp rx=0 tx=0).stop 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 shutdown_connections 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f772c039650 0x7f772c03bb10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 --2- 172.21.15.37:0/2297568409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77401074f0 0x7f774019c480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 >> 172.21.15.37:0/2297568409 conn(0x7f7740102a90 msgr2=0x7f77401038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 shutdown_connections 2024-04-25T13:17:09.136 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:08.516+0000 7f7745930700 1 -- 172.21.15.37:0/2297568409 wait complete. 2024-04-25T13:17:09.137 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-04-25T13:17:10.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:10.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GJUdoW6F5JF20IhfadEMVOZuzdD89pAmGq3qqw9lDB.3tOH36TQDi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714051030, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T13:17:10.878 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.592+0000 7f1bf54ea700 1 Processor -- start 2024-04-25T13:17:10.879 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.592+0000 7f1bf54ea700 1 -- start start 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.592+0000 7f1bf54ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.592+0000 7f1bf54ea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf0063d60 con 0x7f1bf0109040 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.592+0000 7f1beeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1beeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54140/0 (socket says 172.21.15.37:54140) 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1beeffd700 1 -- 172.21.15.37:0/150976163 learned_addr learned my addr 172.21.15.37:0/150976163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:10.880 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1beeffd700 1 -- 172.21.15.37:0/150976163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf010b970 con 0x7f1bf0109040 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1beeffd700 1 --2- 172.21.15.37:0/150976163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098a0 tx=0x7f1be0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454798c5a9e08740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1bedffb700 1 -- 172.21.15.37:0/150976163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0004160 con 0x7f1bf0109040 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1bedffb700 1 -- 172.21.15.37:0/150976163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be0004300 con 0x7f1bf0109040 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.593+0000 7f1bedffb700 1 -- 172.21.15.37:0/150976163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0011a20 con 0x7f1bf0109040 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 -- 172.21.15.37:0/150976163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 msgr2=0x7f1bf010b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/150976163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1be00098a0 tx=0x7f1be0009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 -- 172.21.15.37:0/150976163 shutdown_connections 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/150976163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf010b430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 -- 172.21.15.37:0/150976163 >> 172.21.15.37:0/150976163 conn(0x7f1bf0102ab0 msgr2=0x7f1bf0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 -- 172.21.15.37:0/150976163 shutdown_connections 2024-04-25T13:17:10.893 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.594+0000 7f1bf54ea700 1 -- 172.21.15.37:0/150976163 wait complete. 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1bf54ea700 1 Processor -- start 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1bf54ea700 1 -- start start 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1bf54ea700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1bf54ea700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf010c980 con 0x7f1bf0109040 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1beeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1beeffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54142/0 (socket says 172.21.15.37:54142) 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.595+0000 7f1beeffd700 1 -- 172.21.15.37:0/1570522736 learned_addr learned my addr 172.21.15.37:0/1570522736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1beeffd700 1 -- 172.21.15.37:0/1570522736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf019c980 con 0x7f1bf0109040 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1beeffd700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000c00 tx=0x7f1be0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0003aa0 con 0x7f1bf0109040 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1be001b8f0 con 0x7f1bf0109040 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1be0023870 con 0x7f1bf0109040 2024-04-25T13:17:10.894 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf019cc10 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.596+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf019f900 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.597+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf00640a0 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.598+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f1be0023b20 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.598+0000 7f1bd7fff700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 0x7f1bd803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.598+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1be004fa60 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.598+0000 7f1bee7fc700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 0x7f1bd803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.598+0000 7f1bee7fc700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 0x7f1bd803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1be4006fd0 tx=0x7f1be4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.603+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1be0056070 con 0x7f1bf0109040 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:09.830+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1bf019fb90 con 0x7f1bd8039660 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.078+0000 7f1bd7fff700 1 -- 172.21.15.37:0/1570522736 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1bf019fb90 con 0x7f1bd8039660 2024-04-25T13:17:10.897 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 msgr2=0x7f1bd803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 0x7f1bd803bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1be4006fd0 tx=0x7f1be4006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 msgr2=0x7f1bf019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1be0000c00 tx=0x7f1be0011aa0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 shutdown_connections 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1bd8039660 0x7f1bd803bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 --2- 172.21.15.37:0/1570522736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1bf0109040 0x7f1bf019c440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 >> 172.21.15.37:0/1570522736 conn(0x7f1bf0102ab0 msgr2=0x7f1bf01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 shutdown_connections 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:10.080+0000 7f1bf54ea700 1 -- 172.21.15.37:0/1570522736 wait complete. 2024-04-25T13:17:10.898 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-04-25T13:17:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:11 smithi037 ceph-mon[29204]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:12.342 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:12 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1026347578' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f81268ae700 1 Processor -- start 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f81268ae700 1 -- start start 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f81268ae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f81268ae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f812007c8d0 con 0x7f812007de50 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f811ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f811ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56418/0 (socket says 172.21.15.37:56418) 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f811ffff700 1 -- 172.21.15.37:0/1185651703 learned_addr learned my addr 172.21.15.37:0/1185651703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f811ffff700 1 -- 172.21.15.37:0/1185651703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812007ca50 con 0x7f812007de50 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.771+0000 7f811ffff700 1 --2- 172.21.15.37:0/1185651703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81100098a0 tx=0x7f8110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42ef2f9fbeadfde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f811effd700 1 -- 172.21.15.37:0/1185651703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110004160 con 0x7f812007de50 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f811effd700 1 -- 172.21.15.37:0/1185651703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8110004300 con 0x7f812007de50 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f811effd700 1 -- 172.21.15.37:0/1185651703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110011a20 con 0x7f812007de50 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 -- 172.21.15.37:0/1185651703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 msgr2=0x7f812007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 --2- 172.21.15.37:0/1185651703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81100098a0 tx=0x7f8110009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 -- 172.21.15.37:0/1185651703 shutdown_connections 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 --2- 172.21.15.37:0/1185651703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812007c300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 -- 172.21.15.37:0/1185651703 >> 172.21.15.37:0/1185651703 conn(0x7f8120103bc0 msgr2=0x7f8120105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 -- 172.21.15.37:0/1185651703 shutdown_connections 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.772+0000 7f81268ae700 1 -- 172.21.15.37:0/1185651703 wait complete. 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f81268ae700 1 Processor -- start 2024-04-25T13:17:12.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f81268ae700 1 -- start start 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f81268ae700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f81268ae700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f812010eb80 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f811ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f811ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56430/0 (socket says 172.21.15.37:56430) 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.773+0000 7f811ffff700 1 -- 172.21.15.37:0/1026347578 learned_addr learned my addr 172.21.15.37:0/1026347578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f811ffff700 1 -- 172.21.15.37:0/1026347578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812019eb40 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f811ffff700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8110000c00 tx=0x7f8110011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110003aa0 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f811001b8f0 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812019edd0 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110023870 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.774+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a1ac0 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.775+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8110023b20 con 0x7f812007de50 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.775+0000 7f811d7fa700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 0x7f810803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:12.597 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.775+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81200640a0 con 0x7f812007de50 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.775+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f811004e1b0 con 0x7f812007de50 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.775+0000 7f811f7fe700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 0x7f810803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.776+0000 7f811f7fe700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 0x7f810803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8114006fd0 tx=0x7f8114006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.781+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f811001f030 con 0x7f812007de50 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.987+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f81201a1c90 con 0x7f812007de50 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.987+0000 7f811d7fa700 1 -- 172.21.15.37:0/1026347578 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8110031020 con 0x7f812007de50 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 msgr2=0x7f810803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 0x7f810803bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8114006fd0 tx=0x7f8114006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 msgr2=0x7f812019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8110000c00 tx=0x7f8110011aa0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 shutdown_connections 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8108039660 0x7f810803bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.989+0000 7f81268ae700 1 --2- 172.21.15.37:0/1026347578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812007de50 0x7f812019e600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:12.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.990+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 >> 172.21.15.37:0/1026347578 conn(0x7f8120103bc0 msgr2=0x7f8120104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:12.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.990+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 shutdown_connections 2024-04-25T13:17:12.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:11.990+0000 7f81268ae700 1 -- 172.21.15.37:0/1026347578 wait complete. 2024-04-25T13:17:12.604 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T13:17:12.604 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T13:17:12.612 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T13:17:12.612 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T13:17:12.612 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T13:17:12.613 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T13:17:12.614 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-04-25T13:17:12.614 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:12.614 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037:8443/ 2024-04-25T13:17:12.615 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-04-25T13:17:12.615 INFO:teuthology.orchestra.run.smithi037.stdout: Password: wil57v001h 2024-04-25T13:17:12.615 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:12.615 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config directory 2024-04-25T13:17:12.615 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-04-25T13:17:13.250 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:13 smithi037 ceph-mon[29204]: mgrmap e13: x(active, since 6s) 2024-04-25T13:17:13.899 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.029+0000 7f35dbcca700 1 Processor -- start 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.029+0000 7f35dbcca700 1 -- start start 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.029+0000 7f35dbcca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35dbcca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4063cc0 con 0x7f35d41090e0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d9a66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d9a66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56440/0 (socket says 172.21.15.37:56440) 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d9a66700 1 -- 172.21.15.37:0/3614981193 learned_addr learned my addr 172.21.15.37:0/3614981193 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d9a66700 1 -- 172.21.15.37:0/3614981193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d410ba10 con 0x7f35d41090e0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d9a66700 1 --2- 172.21.15.37:0/3614981193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009e80 tx=0x7f35c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf3cb5ebf8ce369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.030+0000 7f35d8a64700 1 -- 172.21.15.37:0/3614981193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4017070 con 0x7f35d41090e0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35d8a64700 1 -- 172.21.15.37:0/3614981193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c400b8c0 con 0x7f35d41090e0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35d8a64700 1 -- 172.21.15.37:0/3614981193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4004490 con 0x7f35d41090e0 2024-04-25T13:17:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 -- 172.21.15.37:0/3614981193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 msgr2=0x7f35d410b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 --2- 172.21.15.37:0/3614981193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009e80 tx=0x7f35c400b330 comp rx=0 tx=0).stop 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 -- 172.21.15.37:0/3614981193 shutdown_connections 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 --2- 172.21.15.37:0/3614981193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d410b4d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 -- 172.21.15.37:0/3614981193 >> 172.21.15.37:0/3614981193 conn(0x7f35d4102ab0 msgr2=0x7f35d4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 -- 172.21.15.37:0/3614981193 shutdown_connections 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.031+0000 7f35dbcca700 1 -- 172.21.15.37:0/3614981193 wait complete. 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35dbcca700 1 Processor -- start 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35dbcca700 1 -- start start 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35dbcca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35dbcca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35d410c9b0 con 0x7f35d41090e0 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35d9a66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35d9a66700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56442/0 (socket says 172.21.15.37:56442) 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35d9a66700 1 -- 172.21.15.37:0/1619400357 learned_addr learned my addr 172.21.15.37:0/1619400357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.032+0000 7f35d9a66700 1 -- 172.21.15.37:0/1619400357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419ebd0 con 0x7f35d41090e0 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35d9a66700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35c4004220 tx=0x7f35c400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4020020 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c4003b50 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419ee60 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35c4017040 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.033+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a1b50 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.034+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f35c4023020 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.034+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d40640a0 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.034+0000 7f35d2ffd700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 0x7f35bc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.034+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f35c400f050 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.034+0000 7f35d9265700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 0x7f35bc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.035+0000 7f35d9265700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 0x7f35bc03bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.039+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35c402a020 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.250+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f35d41a1d20 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.251+0000 7f35d2ffd700 1 -- 172.21.15.37:0/1619400357 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f35c401b370 con 0x7f35d41090e0 2024-04-25T13:17:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 msgr2=0x7f35bc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 0x7f35bc03bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 msgr2=0x7f35d419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35c4004220 tx=0x7f35c400bd30 comp rx=0 tx=0).stop 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 shutdown_connections 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35bc039650 0x7f35bc03bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 --2- 172.21.15.37:0/1619400357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35d41090e0 0x7f35d419e690 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 >> 172.21.15.37:0/1619400357 conn(0x7f35d4102ab0 msgr2=0x7f35d41034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 shutdown_connections 2024-04-25T13:17:13.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:13.254+0000 7f35dbcca700 1 -- 172.21.15.37:0/1619400357 wait complete. 2024-04-25T13:17:15.238 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.363+0000 7fd76dff5700 1 Processor -- start 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd76dff5700 1 -- start start 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd76dff5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd76dff5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd76807be10 con 0x7fd7681090c0 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56448/0 (socket says 172.21.15.37:56448) 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd7677fe700 1 -- 172.21.15.37:0/4117613245 learned_addr learned my addr 172.21.15.37:0/4117613245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd7677fe700 1 -- 172.21.15.37:0/4117613245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76810b9f0 con 0x7fd7681090c0 2024-04-25T13:17:15.242 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd7677fe700 1 --2- 172.21.15.37:0/4117613245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7580098a0 tx=0x7fd758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b44e91d2535d8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd766ffd700 1 -- 172.21.15.37:0/4117613245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd758004160 con 0x7fd7681090c0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.364+0000 7fd766ffd700 1 -- 172.21.15.37:0/4117613245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd758004300 con 0x7fd7681090c0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd766ffd700 1 -- 172.21.15.37:0/4117613245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd758011a20 con 0x7fd7681090c0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 -- 172.21.15.37:0/4117613245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 msgr2=0x7fd76810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 --2- 172.21.15.37:0/4117613245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd7580098a0 tx=0x7fd758009bb0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 -- 172.21.15.37:0/4117613245 shutdown_connections 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 --2- 172.21.15.37:0/4117613245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76810b4b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 -- 172.21.15.37:0/4117613245 >> 172.21.15.37:0/4117613245 conn(0x7fd768102ad0 msgr2=0x7fd768104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 -- 172.21.15.37:0/4117613245 shutdown_connections 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.365+0000 7fd76dff5700 1 -- 172.21.15.37:0/4117613245 wait complete. 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd76dff5700 1 Processor -- start 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd76dff5700 1 -- start start 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd76dff5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd76dff5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810c990 con 0x7fd7681090c0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd7677fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56454/0 (socket says 172.21.15.37:56454) 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.366+0000 7fd7677fe700 1 -- 172.21.15.37:0/880399365 learned_addr learned my addr 172.21.15.37:0/880399365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd7677fe700 1 -- 172.21.15.37:0/880399365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76819eb30 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd7677fe700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd758000c00 tx=0x7fd758011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd758003aa0 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd75801b8f0 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76819edc0 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd758023870 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.367+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a1ab0 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.368+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd758023b20 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.368+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7680640a0 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.368+0000 7fd7657fa700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 0x7fd75403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.368+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd75804e150 con 0x7fd7681090c0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.368+0000 7fd75effd700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 0x7fd75403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.369+0000 7fd75effd700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 0x7fd75403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd750006fd0 tx=0x7fd750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:15.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.373+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd75801f030 con 0x7fd7681090c0 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.698+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd7681a1c80 con 0x7fd7681090c0 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.701+0000 7fd7657fa700 1 -- 172.21.15.37:0/880399365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fd758023e10 con 0x7fd7681090c0 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 msgr2=0x7fd75403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 0x7fd75403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd750006fd0 tx=0x7fd750006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 msgr2=0x7fd76819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd758000c00 tx=0x7fd758011aa0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 shutdown_connections 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd754039700 0x7fd75403bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 --2- 172.21.15.37:0/880399365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7681090c0 0x7fd76819e5f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 >> 172.21.15.37:0/880399365 conn(0x7fd768102ad0 msgr2=0x7fd7681034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 shutdown_connections 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T13:17:14.703+0000 7fd76dff5700 1 -- 172.21.15.37:0/880399365 wait complete. 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:15.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:15.246 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-04-25T13:17:15.278 INFO:tasks.cephadm:Fetching config... 2024-04-25T13:17:15.279 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:17:15.279 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T13:17:15.294 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T13:17:15.295 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:17:15.295 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T13:17:15.351 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T13:17:15.351 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:17:15.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T13:17:15.415 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T13:17:15.415 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:17:15.416 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T13:17:15.471 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T13:17:15.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:17:15.544 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:17:15.558 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:17:15.598 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:17:15.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T13:17:15.650 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGUpWLvupBsOXRZ7S4wXx2hh4eoJi89H7F6vJg0UK+RDLEzplnx1fsLLP3D67wsZ2CjhCAcp0ROCbOT2vZsAxDLGCFjOIXcFj+ENLj7si4rru1xEkQUc8SHt44YqgCxgR6FcBUuZZhZFM6t2pGCLxd2M0RhlzWU5523dPOExAOtM2TyvzAuRstQbyZbKJu7+g2WyQKUR8IgHm8L+Av26szljgNb9Iyxwu77Z7XznykUHCJ5BuG9irsYtzDzSnnjdYL11Iy5N+56hqj2kj5zuhVCM5odcuXczvTHUz20PghdD4KeKs76zWxeDcwI2yZBr3NXREmtR/AwT59Do8Aai1E5ogDihZdWPoFKXtgx0BRDhbROmm6PFE0GHEGhbKKCBoh2fa0cj0oKWFh5eW8FcnFN7r6u9lfGtPP368MQAUuprhz29yE5H4GWKtgV8hWmFhGkSqAi5/EqtH5POlB3Rt39LUm0xNrCVoR7Gvmy+y8oBbu9eDL12Af6htMOk9lScU= ceph-d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:17:15.660 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T13:17:15.907 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:16.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/880399365' entity='client.admin' 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:18.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 -- 172.21.15.37:0/829959920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 msgr2=0x7f84401017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 --2- 172.21.15.37:0/829959920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f84401017d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8430009910 tx=0x7f8430009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 -- 172.21.15.37:0/829959920 shutdown_connections 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 --2- 172.21.15.37:0/829959920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f84401017d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 -- 172.21.15.37:0/829959920 >> 172.21.15.37:0/829959920 conn(0x7f84400fc950 msgr2=0x7f84400fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 -- 172.21.15.37:0/829959920 shutdown_connections 2024-04-25T13:17:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 -- 172.21.15.37:0/829959920 wait complete. 2024-04-25T13:17:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.584+0000 7f8445ea5700 1 Processor -- start 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f8445ea5700 1 -- start start 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f8445ea5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f8445ea5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f84401072f0 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f843f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56458/0 (socket says 172.21.15.37:56458) 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f843f7fe700 1 -- 172.21.15.37:0/2682965906 learned_addr learned my addr 172.21.15.37:0/2682965906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f843f7fe700 1 -- 172.21.15.37:0/2682965906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8440198010 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.585+0000 7f843f7fe700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8430005800 tx=0x7f8430009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.586+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430027070 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.586+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843000be10 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.586+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f843000f460 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.586+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401982a0 con 0x7f84401013b0 2024-04-25T13:17:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.586+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844019af90 con 0x7f84401013b0 2024-04-25T13:17:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.587+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440107a70 con 0x7f84401013b0 2024-04-25T13:17:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.587+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f843002a070 con 0x7f84401013b0 2024-04-25T13:17:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.587+0000 7f843cff9700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 0x7f842803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.587+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f843001e030 con 0x7f84401013b0 2024-04-25T13:17:18.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.587+0000 7f843effd700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 0x7f842803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:18.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.588+0000 7f843effd700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 0x7f842803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:18.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.592+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8430013070 con 0x7f84401013b0 2024-04-25T13:17:18.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.830+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f844002dff0 con 0x7f84401013b0 2024-04-25T13:17:18.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.833+0000 7f843cff9700 1 -- 172.21.15.37:0/2682965906 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f84300310c0 con 0x7f84401013b0 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.835+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 msgr2=0x7f842803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.835+0000 7f8445ea5700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 0x7f842803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 msgr2=0x7f8440197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8430005800 tx=0x7f8430009f60 comp rx=0 tx=0).stop 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 shutdown_connections 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8428039650 0x7f842803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 --2- 172.21.15.37:0/2682965906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84401013b0 0x7f8440197ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 >> 172.21.15.37:0/2682965906 conn(0x7f84400fc950 msgr2=0x7f844006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 shutdown_connections 2024-04-25T13:17:18.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:18.836+0000 7f8445ea5700 1 -- 172.21.15.37:0/2682965906 wait complete. 2024-04-25T13:17:20.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:19 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2682965906' entity='client.admin' 2024-04-25T13:17:20.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:20.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:20.109 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T13:17:20.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T13:17:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 -- 172.21.15.37:0/633749871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 msgr2=0x7f95341053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 --2- 172.21.15.37:0/633749871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f95341053e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9524009910 tx=0x7f9524009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 -- 172.21.15.37:0/633749871 shutdown_connections 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 --2- 172.21.15.37:0/633749871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f95341053e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 -- 172.21.15.37:0/633749871 >> 172.21.15.37:0/633749871 conn(0x7f95340fc940 msgr2=0x7f95340fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:21.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 -- 172.21.15.37:0/633749871 shutdown_connections 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.954+0000 7f953aecd700 1 -- 172.21.15.37:0/633749871 wait complete. 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f953aecd700 1 Processor -- start 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f953aecd700 1 -- start start 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f953aecd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f953aecd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95341083e0 con 0x7f9534102ff0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f9538c69700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f9538c69700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56708/0 (socket says 172.21.15.37:56708) 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f9538c69700 1 -- 172.21.15.37:0/2531407535 learned_addr learned my addr 172.21.15.37:0/2531407535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f9538c69700 1 -- 172.21.15.37:0/2531407535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9534198030 con 0x7f9534102ff0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.955+0000 7f9538c69700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9524005b40 tx=0x7f9524004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.956+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524027070 con 0x7f9534102ff0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.956+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341982c0 con 0x7f9534102ff0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.956+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952400be10 con 0x7f9534102ff0 2024-04-25T13:17:21.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.956+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9524022de0 con 0x7f9534102ff0 2024-04-25T13:17:21.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.956+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953419af60 con 0x7f9534102ff0 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.957+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f952402a070 con 0x7f9534102ff0 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.957+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534108b60 con 0x7f9534102ff0 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.957+0000 7f9531ffb700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 0x7f951c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.957+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f952401e030 con 0x7f9534102ff0 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.957+0000 7f9533fff700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 0x7f951c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:21.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.958+0000 7f9533fff700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 0x7f951c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:21.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:21.962+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9524051030 con 0x7f9534102ff0 2024-04-25T13:17:22.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.173+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f953402e120 con 0x7f951c039650 2024-04-25T13:17:22.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.176+0000 7f9531ffb700 1 -- 172.21.15.37:0/2531407535 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f953402e120 con 0x7f951c039650 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 msgr2=0x7f951c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 0x7f951c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9528006fd0 tx=0x7f9528006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 msgr2=0x7f9534197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9524005b40 tx=0x7f9524004f80 comp rx=0 tx=0).stop 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 shutdown_connections 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f951c039650 0x7f951c03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 --2- 172.21.15.37:0/2531407535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9534102ff0 0x7f9534197af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.178+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 >> 172.21.15.37:0/2531407535 conn(0x7f95340fc940 msgr2=0x7f95340fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.179+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 shutdown_connections 2024-04-25T13:17:22.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:22.179+0000 7f953aecd700 1 -- 172.21.15.37:0/2531407535 wait complete. 2024-04-25T13:17:22.961 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-04-25T13:17:22.961 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:17:22.961 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:17:22.981 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:17:22.981 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:23.040 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-04-25T13:17:23.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch host add smithi044 2024-04-25T13:17:23.285 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:17:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:23.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:23.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:24 smithi037 ceph-mon[29204]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:24.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:24 smithi037 ceph-mon[29204]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:17:24.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 -- 172.21.15.37:0/1518105900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 msgr2=0x7fbc64108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 --2- 172.21.15.37:0/1518105900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64108930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009910 tx=0x7fbc58009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 -- 172.21.15.37:0/1518105900 shutdown_connections 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 --2- 172.21.15.37:0/1518105900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64108930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 -- 172.21.15.37:0/1518105900 >> 172.21.15.37:0/1518105900 conn(0x7fbc640fdf70 msgr2=0x7fbc64100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 -- 172.21.15.37:0/1518105900 shutdown_connections 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.805+0000 7fbc69338700 1 -- 172.21.15.37:0/1518105900 wait complete. 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.806+0000 7fbc69338700 1 Processor -- start 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.806+0000 7fbc69338700 1 -- start start 2024-04-25T13:17:24.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.806+0000 7fbc69338700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc69338700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6410c9c0 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc62ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc62ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56730/0 (socket says 172.21.15.37:56730) 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc62ffd700 1 -- 172.21.15.37:0/2924016013 learned_addr learned my addr 172.21.15.37:0/2924016013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc62ffd700 1 -- 172.21.15.37:0/2924016013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc64076c30 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc62ffd700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc580056d0 tx=0x7fbc580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc58026070 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc64075280 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc5800bc00 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc58003920 con 0x7fbc64108550 2024-04-25T13:17:24.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.807+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc64075740 con 0x7fbc64108550 2024-04-25T13:17:24.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.808+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbc58029070 con 0x7fbc64108550 2024-04-25T13:17:24.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.808+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc64103bc0 con 0x7fbc64108550 2024-04-25T13:17:24.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.808+0000 7fbc4bfff700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 0x7fbc4c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:24.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.808+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc5801e070 con 0x7fbc64108550 2024-04-25T13:17:24.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.809+0000 7fbc627fc700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 0x7fbc4c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:24.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.809+0000 7fbc627fc700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 0x7fbc4c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006fd0 tx=0x7fbc54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:24.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:24.813+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc58013070 con 0x7fbc64108550 2024-04-25T13:17:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:25.027+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) -- 0x7fbc64075970 con 0x7fbc4c039650 2024-04-25T13:17:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:25 smithi037 ceph-mon[29204]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:26 smithi037 ceph-mon[29204]: Deploying cephadm binary to smithi044 2024-04-25T13:17:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:26 smithi037 ceph-mon[29204]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:28 smithi037 ceph-mon[29204]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:29.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.255+0000 7fbc4bfff700 1 -- 172.21.15.37:0/2924016013 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbc64075970 con 0x7fbc4c039650 2024-04-25T13:17:29.256 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-04-25T13:17:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 msgr2=0x7fbc4c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 0x7fbc4c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006fd0 tx=0x7fbc54006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 msgr2=0x7fbc64074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc580056d0 tx=0x7fbc580057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 shutdown_connections 2024-04-25T13:17:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbc4c039650 0x7fbc4c03bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 --2- 172.21.15.37:0/2924016013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64108550 0x7fbc64074d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 >> 172.21.15.37:0/2924016013 conn(0x7fbc640fdf70 msgr2=0x7fbc640fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 shutdown_connections 2024-04-25T13:17:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:29.258+0000 7fbc69338700 1 -- 172.21.15.37:0/2924016013 wait complete. 2024-04-25T13:17:29.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:17:30.137 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:30.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:30.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:30 smithi037 ceph-mon[29204]: Added host smithi044 2024-04-25T13:17:30.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:30.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:31.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:31 smithi037 ceph-mon[29204]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:31.645 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 -- 172.21.15.37:0/3549106470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 msgr2=0x7fbe6c1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3549106470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c1019d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009910 tx=0x7fbe5c009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 -- 172.21.15.37:0/3549106470 shutdown_connections 2024-04-25T13:17:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3549106470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c1019d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 -- 172.21.15.37:0/3549106470 >> 172.21.15.37:0/3549106470 conn(0x7fbe6c0fcea0 msgr2=0x7fbe6c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 -- 172.21.15.37:0/3549106470 shutdown_connections 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.751+0000 7fbe7194f700 1 -- 172.21.15.37:0/3549106470 wait complete. 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe7194f700 1 Processor -- start 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe7194f700 1 -- start start 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe7194f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe7194f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10b870 con 0x7fbe6c1015f0 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe6affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe6affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41950/0 (socket says 172.21.15.37:41950) 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.752+0000 7fbe6affd700 1 -- 172.21.15.37:0/3081096155 learned_addr learned my addr 172.21.15.37:0/3081096155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe6affd700 1 -- 172.21.15.37:0/3081096155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c198420 con 0x7fbe6c1015f0 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe6affd700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0056d0 tx=0x7fbe5c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe5c026070 con 0x7fbe6c1015f0 2024-04-25T13:17:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c00bba0 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c198710 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe5c003920 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.753+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c19c7b0 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.754+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbe5c029070 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.754+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c102ad0 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.754+0000 7fbe7094d700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 0x7fbe5403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.754+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe5c01e070 con 0x7fbe6c1015f0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.754+0000 7fbe6a7fc700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 0x7fbe5403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.755+0000 7fbe6a7fc700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 0x7fbe5403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006fd0 tx=0x7fbe60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:31.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.759+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe5c013070 con 0x7fbe6c1015f0 2024-04-25T13:17:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.975+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe6c0640a0 con 0x7fbe540396b0 2024-04-25T13:17:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.976+0000 7fbe7094d700 1 -- 172.21.15.37:0/3081096155 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fbe6c0640a0 con 0x7fbe540396b0 2024-04-25T13:17:31.976 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:31.976 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-04-25T13:17:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 msgr2=0x7fbe5403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 0x7fbe5403bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006fd0 tx=0x7fbe60006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 msgr2=0x7fbe6c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0056d0 tx=0x7fbe5c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 shutdown_connections 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbe540396b0 0x7fbe5403bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 --2- 172.21.15.37:0/3081096155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe6c1015f0 0x7fbe6c197ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 >> 172.21.15.37:0/3081096155 conn(0x7fbe6c0fcea0 msgr2=0x7fbe6c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 shutdown_connections 2024-04-25T13:17:31.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:31.978+0000 7fbe7194f700 1 -- 172.21.15.37:0/3081096155 wait complete. 2024-04-25T13:17:32.614 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi069 2024-04-25T13:17:32.614 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:17:32.614 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.conf 2024-04-25T13:17:32.634 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:17:32.634 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:32.691 INFO:tasks.cephadm:Adding host smithi069 to orchestrator... 2024-04-25T13:17:32.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch host add smithi069 2024-04-25T13:17:32.937 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:33.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:32 smithi037 ceph-mon[29204]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:33.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:32 smithi037 ceph-mon[29204]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:33.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 -- 172.21.15.37:0/1316878452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 msgr2=0x7fa8c0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 --2- 172.21.15.37:0/1316878452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0103a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009910 tx=0x7fa8b0009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 -- 172.21.15.37:0/1316878452 shutdown_connections 2024-04-25T13:17:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 --2- 172.21.15.37:0/1316878452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0103a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 -- 172.21.15.37:0/1316878452 >> 172.21.15.37:0/1316878452 conn(0x7fa8c00fcea0 msgr2=0x7fa8c00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 -- 172.21.15.37:0/1316878452 shutdown_connections 2024-04-25T13:17:34.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.515+0000 7fa8c5078700 1 -- 172.21.15.37:0/1316878452 wait complete. 2024-04-25T13:17:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.516+0000 7fa8c5078700 1 Processor -- start 2024-04-25T13:17:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.516+0000 7fa8c5078700 1 -- start start 2024-04-25T13:17:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.516+0000 7fa8c5078700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.516+0000 7fa8c5078700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c010c9d0 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8bed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8bed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41962/0 (socket says 172.21.15.37:41962) 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8bed9d700 1 -- 172.21.15.37:0/2048011787 learned_addr learned my addr 172.21.15.37:0/2048011787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8bed9d700 1 -- 172.21.15.37:0/2048011787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c01984b0 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8bed9d700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00056d0 tx=0x7fa8b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b0026070 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c0198740 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b0004370 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b00039b0 con 0x7fa8c0103650 2024-04-25T13:17:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.517+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c0198ba0 con 0x7fa8c0103650 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.518+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa8b0029070 con 0x7fa8c0103650 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.518+0000 7fa8a7fff700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 0x7fa8a803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.518+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8b001e070 con 0x7fa8c0103650 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.518+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c006b190 con 0x7fa8c0103650 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.519+0000 7fa8be59c700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 0x7fa8a803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.519+0000 7fa8be59c700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 0x7fa8a803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006fd0 tx=0x7fa8b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.524+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8b0030020 con 0x7fa8c0103650 2024-04-25T13:17:34.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:34.735+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}) -- 0x7fa8c01992d0 con 0x7fa8a8039650 2024-04-25T13:17:35.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:34 smithi037 ceph-mon[29204]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:17:36.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:36 smithi037 ceph-mon[29204]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:17:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:37 smithi037 ceph-mon[29204]: Deploying cephadm binary to smithi069 2024-04-25T13:17:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:37 smithi037 ceph-mon[29204]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:37 smithi037 ceph-mon[29204]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:37 smithi037 ceph-mon[29204]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:17:39.191 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi069' with addr '172.21.15.69' 2024-04-25T13:17:39.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.189+0000 7fa8a7fff700 1 -- 172.21.15.37:0/2048011787 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa8c01992d0 con 0x7fa8a8039650 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 msgr2=0x7fa8a803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 0x7fa8a803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4006fd0 tx=0x7fa8b4006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 msgr2=0x7fa8c0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00056d0 tx=0x7fa8b00057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 shutdown_connections 2024-04-25T13:17:39.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8a8039650 0x7fa8a803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 --2- 172.21.15.37:0/2048011787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0103650 0x7fa8c0197f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 >> 172.21.15.37:0/2048011787 conn(0x7fa8c00fcea0 msgr2=0x7fa8c01043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 shutdown_connections 2024-04-25T13:17:39.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:39.192+0000 7fa8c5078700 1 -- 172.21.15.37:0/2048011787 wait complete. 2024-04-25T13:17:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:39 smithi037 ceph-mon[29204]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:39.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T13:17:40.236 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:40.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:40 smithi037 ceph-mon[29204]: Added host smithi069 2024-04-25T13:17:40.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:40.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:41 smithi037 ceph-mon[29204]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:41.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:41.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- 172.21.15.37:0/2702737429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 msgr2=0x7fdfa8108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 --2- 172.21.15.37:0/2702737429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8108950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009910 tx=0x7fdfa4009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- 172.21.15.37:0/2702737429 shutdown_connections 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 --2- 172.21.15.37:0/2702737429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8108950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- 172.21.15.37:0/2702737429 >> 172.21.15.37:0/2702737429 conn(0x7fdfa8078970 msgr2=0x7fdfa8078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- 172.21.15.37:0/2702737429 shutdown_connections 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- 172.21.15.37:0/2702737429 wait complete. 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 Processor -- start 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.916+0000 7fdfb020f700 1 -- start start 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfb020f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfb020f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa810c920 con 0x7fdfa8108570 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfadfab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfadfab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46572/0 (socket says 172.21.15.37:46572) 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfadfab700 1 -- 172.21.15.37:0/3251378094 learned_addr learned my addr 172.21.15.37:0/3251378094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:41.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfadfab700 1 -- 172.21.15.37:0/3251378094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa8198300 con 0x7fdfa8108570 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfadfab700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40056d0 tx=0x7fdfa40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4026070 con 0x7fdfa8108570 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa400bc00 con 0x7fdfa8108570 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa81985f0 con 0x7fdfa8108570 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfa4003920 con 0x7fdfa8108570 2024-04-25T13:17:41.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.917+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa819c640 con 0x7fdfa8108570 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.918+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdfa4029070 con 0x7fdfa8108570 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.918+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa806a620 con 0x7fdfa8108570 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.918+0000 7fdf9effd700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 0x7fdf9403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.918+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdfa401e070 con 0x7fdfa8108570 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.919+0000 7fdfad7aa700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 0x7fdf9403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:41.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.919+0000 7fdfad7aa700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 0x7fdf9403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdf98006fd0 tx=0x7fdf98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:41.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:41.924+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfa81090a0 con 0x7fdfa8108570 2024-04-25T13:17:42.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.138+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfa81990d0 con 0x7fdf94039650 2024-04-25T13:17:42.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.139+0000 7fdf9effd700 1 -- 172.21.15.37:0/3251378094 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7fdfa81990d0 con 0x7fdf94039650 2024-04-25T13:17:42.140 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:17:42.140 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}] 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 msgr2=0x7fdf9403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 0x7fdf9403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdf98006fd0 tx=0x7fdf98006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 msgr2=0x7fdfa8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40056d0 tx=0x7fdfa40057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 shutdown_connections 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdf94039650 0x7fdf9403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 --2- 172.21.15.37:0/3251378094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa8108570 0x7fdfa8197dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:42.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 >> 172.21.15.37:0/3251378094 conn(0x7fdfa8078970 msgr2=0x7fdfa80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:42.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 shutdown_connections 2024-04-25T13:17:42.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:42.141+0000 7fdfb020f700 1 -- 172.21.15.37:0/3251378094 wait complete. 2024-04-25T13:17:42.793 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T13:17:42.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T13:17:43.038 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:17:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:42 smithi037 ceph-mon[29204]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:42 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:42 smithi037 ceph-mon[29204]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.607+0000 7f7760a85700 1 -- 172.21.15.37:0/2114661637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 msgr2=0x7f775c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.607+0000 7f7760a85700 1 --2- 172.21.15.37:0/2114661637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c1088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f774c009910 tx=0x7f774c009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 -- 172.21.15.37:0/2114661637 shutdown_connections 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 --2- 172.21.15.37:0/2114661637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c1088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 -- 172.21.15.37:0/2114661637 >> 172.21.15.37:0/2114661637 conn(0x7f775c0fdf80 msgr2=0x7f775c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 -- 172.21.15.37:0/2114661637 shutdown_connections 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 -- 172.21.15.37:0/2114661637 wait complete. 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 Processor -- start 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.608+0000 7f7760a85700 1 -- start start 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f7760a85700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f7760a85700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f775c10c940 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f775a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f775a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46578/0 (socket says 172.21.15.37:46578) 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f775a59c700 1 -- 172.21.15.37:0/4252848469 learned_addr learned my addr 172.21.15.37:0/4252848469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f775a59c700 1 -- 172.21.15.37:0/4252848469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c1983f0 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f775a59c700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f774c0056d0 tx=0x7f774c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f774c026070 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774c00bba0 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c1986e0 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f774c003920 con 0x7f775c1084e0 2024-04-25T13:17:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.609+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c19c780 con 0x7f775c1084e0 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.610+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f774c029070 con 0x7f775c1084e0 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.610+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775c103b90 con 0x7f775c1084e0 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.611+0000 7f774b7fe700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 0x7f774403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.611+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f774c01e070 con 0x7f775c1084e0 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.611+0000 7f7759d9b700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 0x7f774403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.611+0000 7f7759d9b700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 0x7f774403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7750006fd0 tx=0x7f7750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.616+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f774c013070 con 0x7f775c1084e0 2024-04-25T13:17:44.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:44.831+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f775c199210 con 0x7f775c1084e0 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:45.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.758+0000 7f774b7fe700 1 -- 172.21.15.37:0/4252848469 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f774c030090 con 0x7f775c1084e0 2024-04-25T13:17:45.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 msgr2=0x7f774403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 0x7f774403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7750006fd0 tx=0x7f7750006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 msgr2=0x7f775c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f774c0056d0 tx=0x7f774c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 shutdown_connections 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f77440396b0 0x7f774403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 --2- 172.21.15.37:0/4252848469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f775c1084e0 0x7f775c197eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 >> 172.21.15.37:0/4252848469 conn(0x7f775c0fdf80 msgr2=0x7f775c10a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 shutdown_connections 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:17:45.760+0000 7f7760a85700 1 -- 172.21.15.37:0/4252848469 wait complete. 2024-04-25T13:17:45.762 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4252848469' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:46.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:46.373 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-04-25T13:17:46.373 INFO:tasks.cephadm:Adding mon.b on smithi044 2024-04-25T13:17:46.373 INFO:tasks.cephadm:Adding mon.c on smithi069 2024-04-25T13:17:46.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi044:172.21.15.44=b;smithi069:172.21.15.69=c' 2024-04-25T13:17:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:17:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:46 smithi037 ceph-mon[29204]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:46 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4252848469' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T13:17:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:46 smithi037 ceph-mon[29204]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 -- 172.21.15.69:0/2071954871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 msgr2=0x7f146c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 --2- 172.21.15.69:0/2071954871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c1088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f145c009910 tx=0x7f145c009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 -- 172.21.15.69:0/2071954871 shutdown_connections 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 --2- 172.21.15.69:0/2071954871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c1088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 -- 172.21.15.69:0/2071954871 >> 172.21.15.69:0/2071954871 conn(0x7f146c0fdf80 msgr2=0x7f146c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 -- 172.21.15.69:0/2071954871 shutdown_connections 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.000+0000 7f1471856700 1 -- 172.21.15.69:0/2071954871 wait complete. 2024-04-25T13:17:48.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.001+0000 7f1471856700 1 Processor -- start 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.001+0000 7f1471856700 1 -- start start 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.001+0000 7f1471856700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.001+0000 7f1471856700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f146c10c940 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f146affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f146affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.69:47984/0 (socket says 172.21.15.69:47984) 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f146affd700 1 -- 172.21.15.69:0/2833540871 learned_addr learned my addr 172.21.15.69:0/2833540871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f146affd700 1 -- 172.21.15.69:0/2833540871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c1983f0 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f146affd700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f145c0056d0 tx=0x7f145c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c026070 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c1986e0 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145c00bba0 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c003920 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.002+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c19c780 con 0x7f146c1084e0 2024-04-25T13:17:48.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.003+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f145c029070 con 0x7f146c1084e0 2024-04-25T13:17:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.003+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146c103b90 con 0x7f146c1084e0 2024-04-25T13:17:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.003+0000 7f1470854700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 0x7f145803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.003+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f145c01e070 con 0x7f146c1084e0 2024-04-25T13:17:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.004+0000 7f14627fc700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 0x7f145803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:48.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.004+0000 7f14627fc700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 0x7f145803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.009+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f145c013070 con 0x7f146c1084e0 2024-04-25T13:17:48.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.221+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi044:172.21.15.44=b;smithi069:172.21.15.69=c", "target": ["mon-mgr", ""]}) -- 0x7f146c199210 con 0x7f1458039650 2024-04-25T13:17:48.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.228+0000 7f1470854700 1 -- 172.21.15.69:0/2833540871 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f146c199210 con 0x7f1458039650 2024-04-25T13:17:48.229 INFO:teuthology.orchestra.run.smithi069.stdout:Scheduled mon update... 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 msgr2=0x7f145803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 0x7f145803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1454006fd0 tx=0x7f1454006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 msgr2=0x7f146c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f145c0056d0 tx=0x7f145c0057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 shutdown_connections 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1458039650 0x7f145803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 --2- 172.21.15.69:0/2833540871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f146c1084e0 0x7f146c197eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:48.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 >> 172.21.15.69:0/2833540871 conn(0x7f146c0fdf80 msgr2=0x7f146c10a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:48.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 shutdown_connections 2024-04-25T13:17:48.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:48.230+0000 7f1471856700 1 -- 172.21.15.69:0/2833540871 wait complete. 2024-04-25T13:17:48.839 DEBUG:teuthology.orchestra.run.smithi044:mon.b> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T13:17:48.842 DEBUG:teuthology.orchestra.run.smithi069:mon.c> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T13:17:48.844 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:17:48.844 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi044:172.21.15.44=b;smithi069:172.21.15.69=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi044:172.21.15.44=b;smithi069:172.21.15.69=c;count:3 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:49.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:17:49.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:49.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:49 smithi037 ceph-mon[29204]: Deploying daemon mon.c on smithi069 2024-04-25T13:17:50.541 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:17:51.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:51 smithi037 ceph-mon[29204]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.641+0000 7f1eed1c2700 1 -- 172.21.15.69:0/3799915645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 msgr2=0x7f1ee81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.641+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/3799915645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee81088c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009910 tx=0x7f1ed8009c20 comp rx=0 tx=0).stop 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.641+0000 7f1eed1c2700 1 -- 172.21.15.69:0/3799915645 shutdown_connections 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.641+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/3799915645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee81088c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.641+0000 7f1eed1c2700 1 -- 172.21.15.69:0/3799915645 >> 172.21.15.69:0/3799915645 conn(0x7f1ee80fdf80 msgr2=0x7f1ee81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.642+0000 7f1eed1c2700 1 -- 172.21.15.69:0/3799915645 shutdown_connections 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.642+0000 7f1eed1c2700 1 -- 172.21.15.69:0/3799915645 wait complete. 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.642+0000 7f1eed1c2700 1 Processor -- start 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.642+0000 7f1eed1c2700 1 -- start start 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1eed1c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1eed1c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810c940 con 0x7f1ee81084e0 2024-04-25T13:17:52.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1ee6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1ee6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.69:53668/0 (socket says 172.21.15.69:53668) 2024-04-25T13:17:52.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1ee6d9d700 1 -- 172.21.15.69:0/1839449619 learned_addr learned my addr 172.21.15.69:0/1839449619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:17:52.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1ee6d9d700 1 -- 172.21.15.69:0/1839449619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee81983f0 con 0x7f1ee81084e0 2024-04-25T13:17:52.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.643+0000 7f1ee6d9d700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80056d0 tx=0x7f1ed80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.644+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8026070 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.644+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81986e0 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.644+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed800bba0 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.644+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8003920 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.644+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee819c780 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.645+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee8103b90 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.645+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ed8029070 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.645+0000 7f1ecffff700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 0x7f1ed003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.645+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1ed801e070 con 0x7f1ee81084e0 2024-04-25T13:17:52.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.645+0000 7f1ee659c700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 0x7f1ed003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:52.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.646+0000 7f1ee659c700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 0x7f1ed003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1edc006fd0 tx=0x7f1edc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.650+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ed8013070 con 0x7f1ee81084e0 2024-04-25T13:17:52.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.951+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1ee8199210 con 0x7f1ee81084e0 2024-04-25T13:17:52.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.951+0000 7f1ecffff700 1 -- 172.21.15.69:0/1839449619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1ed80282c0 con 0x7f1ee81084e0 2024-04-25T13:17:52.953 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:17:52.953 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","modified":"2024-04-25T13:15:55.450143Z","created":"2024-04-25T13:15:55.450143Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 msgr2=0x7f1ed003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 0x7f1ed003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1edc006fd0 tx=0x7f1edc006e40 comp rx=0 tx=0).stop 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 msgr2=0x7f1ee8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80056d0 tx=0x7f1ed80057b0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 shutdown_connections 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ed0039650 0x7f1ed003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 --2- 172.21.15.69:0/1839449619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee81084e0 0x7f1ee8197eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 >> 172.21.15.69:0/1839449619 conn(0x7f1ee80fdf80 msgr2=0x7f1ee810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:52.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.953+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 shutdown_connections 2024-04-25T13:17:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:52.954+0000 7f1eed1c2700 1 -- 172.21.15.69:0/1839449619 wait complete. 2024-04-25T13:17:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-04-25T13:17:54.716 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:17:54.716 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:17:54.959 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:17:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:17:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: mon.a calling monitor election 2024-04-25T13:17:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: mon.c calling monitor election 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: monmap epoch 2 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: last_changed 2024-04-25T13:17:52.970678+0000 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: min_mon_release 19 (squid) 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: election_strategy: 1 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: 1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.c 2024-04-25T13:17:58.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: fsmap 2024-04-25T13:17:58.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:17:58.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: mgrmap e13: x(active, since 52s) 2024-04-25T13:17:58.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: overall HEALTH_OK 2024-04-25T13:17:58.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:58.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 -- 172.21.15.69:0/3859902707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c103710 msgr2=0x7ffb78005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 --2- 172.21.15.69:0/3859902707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c103710 0x7ffb78005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb88005fa0 tx=0x7ffb8800ffc0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 -- 172.21.15.69:0/3859902707 shutdown_connections 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 --2- 172.21.15.69:0/3859902707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c103710 0x7ffb78005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 -- 172.21.15.69:0/3859902707 >> 172.21.15.69:0/3859902707 conn(0x7ffb8c0fcf80 msgr2=0x7ffb8c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 -- 172.21.15.69:0/3859902707 shutdown_connections 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.292+0000 7ffb944ca700 1 -- 172.21.15.69:0/3859902707 wait complete. 2024-04-25T13:17:59.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.293+0000 7ffb944ca700 1 Processor -- start 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.293+0000 7ffb944ca700 1 -- start start 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb944ca700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb944ca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 0x7ffb8c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb944ca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c10a4a0 con 0x7ffb8c198550 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb944ca700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c10a320 con 0x7ffb8c103710 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48698/0 (socket says 172.21.15.69:48698) 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 -- 172.21.15.69:0/494213519 learned_addr learned my addr 172.21.15.69:0/494213519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 -- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 msgr2=0x7ffb8c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 -- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 msgr2=0x7ffb8c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb91a65700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 0x7ffb8c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb92266700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:17:59.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb91a65700 1 -- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 msgr2=0x7ffb8c198010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:17:59.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb91a65700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.294+0000 7ffb91a65700 1 -- 172.21.15.69:0/494213519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c198b90 con 0x7ffb8c198550 2024-04-25T13:17:59.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb91a65700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 0x7ffb8c19c8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00b4d0 tx=0x7ffb7c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffb7c011630 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c19ce20 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb7c011c80 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ffb7c010da0 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.295+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c19d380 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.296+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffb7c018020 con 0x7ffb8c198550 2024-04-25T13:17:59.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.296+0000 7ffb837fe700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 0x7ffb7003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:17:59.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.297+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffb7c04c190 con 0x7ffb8c198550 2024-04-25T13:17:59.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.297+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c06b530 con 0x7ffb8c198550 2024-04-25T13:17:59.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.297+0000 7ffb92266700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 0x7ffb7003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:17:59.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.297+0000 7ffb92266700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 0x7ffb7003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800fb20 tx=0x7ffb8801d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:17:59.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.302+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb7c01f020 con 0x7ffb8c198550 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: Deploying daemon mon.b on smithi044 2024-04-25T13:17:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:17:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:17:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.583+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffb8c199370 con 0x7ffb8c198550 2024-04-25T13:17:59.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.584+0000 7ffb837fe700 1 -- 172.21.15.69:0/494213519 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7ffb7c010410 con 0x7ffb8c198550 2024-04-25T13:17:59.584 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:17:59.585 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","modified":"2024-04-25T13:17:52.970678Z","created":"2024-04-25T13:15:55.450143Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 msgr2=0x7ffb7003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 0x7ffb7003b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800fb20 tx=0x7ffb8801d040 comp rx=0 tx=0).stop 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 msgr2=0x7ffb8c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 0x7ffb8c19c8e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00b4d0 tx=0x7ffb7c00b890 comp rx=0 tx=0).stop 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 shutdown_connections 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffb700393f0 0x7ffb7003b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb8c103710 0x7ffb8c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 --2- 172.21.15.69:0/494213519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb8c198550 0x7ffb8c19c8e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:17:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.586+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 >> 172.21.15.69:0/494213519 conn(0x7ffb8c0fcf80 msgr2=0x7ffb8c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:17:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.587+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 shutdown_connections 2024-04-25T13:17:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:17:59.587+0000 7ffb944ca700 1 -- 172.21.15.69:0/494213519 wait complete. 2024-04-25T13:17:59.588 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 2 2024-04-25T13:18:00.346 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/494213519' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:18:01.148 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T13:18:01.148 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T13:18:01.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:01 smithi037 ceph-mon[29204]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:01.388 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:18:02.042 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:02 smithi044 ceph-mon[32590]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T13:18:02.368 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:02 smithi044 bash[32569]: fff21b0440d5ab0ec3621346ebc5a7d2cfe761f98c4953ca2ac0aa337364bee2 2024-04-25T13:18:02.368 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:02 smithi044 systemd[1]: Started Ceph mon.b for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 -- 172.21.15.69:0/3918332681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f40035c0 msgr2=0x7fb2f4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 --2- 172.21.15.69:0/3918332681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f40035c0 0x7fb2f4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb30c074c90 tx=0x7fb2fc00fe60 comp rx=0 tx=0).stop 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 -- 172.21.15.69:0/3918332681 shutdown_connections 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 --2- 172.21.15.69:0/3918332681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f40035c0 0x7fb2f4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 -- 172.21.15.69:0/3918332681 >> 172.21.15.69:0/3918332681 conn(0x7fb30c0fe050 msgr2=0x7fb30c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 -- 172.21.15.69:0/3918332681 shutdown_connections 2024-04-25T13:18:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.174+0000 7fb3125d6700 1 -- 172.21.15.69:0/3918332681 wait complete. 2024-04-25T13:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.175+0000 7fb3125d6700 1 Processor -- start 2024-04-25T13:18:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.175+0000 7fb3125d6700 1 -- start start 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 0x7fb30c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10c660 con 0x7fb30c19a850 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10c4e0 con 0x7fb30c19f120 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb3125d6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c10c7e0 con 0x7fb2f40035c0 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37842/0 (socket says 172.21.15.69:37842) 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 -- 172.21.15.69:0/4270953844 learned_addr learned my addr 172.21.15.69:0/4270953844 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb310b73700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 msgr2=0x7fb30c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 msgr2=0x7fb30c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:18:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30b7fe700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 0x7fb30c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb30bfff700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb310b73700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 msgr2=0x7fb30c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb310b73700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 msgr2=0x7fb30c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb310b73700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.176+0000 7fb310b73700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 msgr2=0x7fb30c19a310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 msgr2=0x7fb30c19f5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 -- 172.21.15.69:0/4270953844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c1a6e90 con 0x7fb30c19a850 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb30b7fe700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 0x7fb30c19ebe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb30000b550 tx=0x7fb30000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004230 con 0x7fb30c19a850 2024-04-25T13:18:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3000043d0 con 0x7fb30c19a850 2024-04-25T13:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300011b90 con 0x7fb30c19a850 2024-04-25T13:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c1a70c0 con 0x7fb30c19a850 2024-04-25T13:18:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.177+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a7650 con 0x7fb30c19a850 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.178+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb30001a400 con 0x7fb30c19a850 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.179+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c0640a0 con 0x7fb30c19a850 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.179+0000 7fb3097fa700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 0x7fb2f803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.179+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb30004c660 con 0x7fb30c19a850 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.179+0000 7fb30bfff700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 0x7fb2f803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.180+0000 7fb30bfff700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 0x7fb2f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc005d10 tx=0x7fb2fc005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.184+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb30001a840 con 0x7fb30c19a850 2024-04-25T13:18:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:18:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: mon.a calling monitor election 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: mon.c calling monitor election 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: monmap epoch 3 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: last_changed 2024-04-25T13:18:02.020944+0000 2024-04-25T13:18:07.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: min_mon_release 19 (squid) 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: election_strategy: 1 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: 1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.c 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: fsmap 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: mgrmap e13: x(active, since 61s) 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: overall HEALTH_OK 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:07.353 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:07.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.483+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb30c02e140 con 0x7fb30c19a850 2024-04-25T13:18:07.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.483+0000 7fb3097fa700 1 -- 172.21.15.69:0/4270953844 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fb300018510 con 0x7fb30c19a850 2024-04-25T13:18:07.484 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-04-25T13:18:07.484 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":3,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","modified":"2024-04-25T13:18:02.020944Z","created":"2024-04-25T13:15:55.450143Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T13:18:07.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 msgr2=0x7fb2f803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:07.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 0x7fb2f803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc005d10 tx=0x7fb2fc005ca0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 msgr2=0x7fb30c19ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 0x7fb30c19ebe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb30000b550 tx=0x7fb30000b910 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 shutdown_connections 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2f8039ae0 0x7fb2f803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb30c19f120 0x7fb30c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f40035c0 0x7fb30c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 --2- 172.21.15.69:0/4270953844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb30c19a850 0x7fb30c19ebe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 >> 172.21.15.69:0/4270953844 conn(0x7fb30c0fe050 msgr2=0x7fb30c109880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 shutdown_connections 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:07.486+0000 7fb3125d6700 1 -- 172.21.15.69:0/4270953844 wait complete. 2024-04-25T13:18:07.487 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 3 2024-04-25T13:18:08.107 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T13:18:08.108 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T13:18:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:08 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/4270953844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:18:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:08.365 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.a calling monitor election 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.c calling monitor election 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:09.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: monmap epoch 3 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: last_changed 2024-04-25T13:18:02.020944+0000 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: min_mon_release 19 (squid) 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: election_strategy: 1 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.c 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: fsmap 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mgrmap e13: x(active, since 61s) 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: overall HEALTH_OK 2024-04-25T13:18:09.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/4270953844' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T13:18:09.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:10.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 -- 172.21.15.37:0/545117610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 msgr2=0x7fa7241088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 --2- 172.21.15.37:0/545117610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa7241088c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa714009910 tx=0x7fa714009c20 comp rx=0 tx=0).stop 2024-04-25T13:18:10.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 -- 172.21.15.37:0/545117610 shutdown_connections 2024-04-25T13:18:10.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 --2- 172.21.15.37:0/545117610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa7241088c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 -- 172.21.15.37:0/545117610 >> 172.21.15.37:0/545117610 conn(0x7fa724076ea0 msgr2=0x7fa7240772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 -- 172.21.15.37:0/545117610 shutdown_connections 2024-04-25T13:18:10.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.042+0000 7fa72933c700 1 -- 172.21.15.37:0/545117610 wait complete. 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 Processor -- start 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 -- start start 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7241019a0 0x7fa724101e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7241a42c0 0x7fa7241a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410c920 con 0x7fa7241084e0 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410c7a0 con 0x7fa7241a42c0 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa72933c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410caa0 con 0x7fa7241019a0 2024-04-25T13:18:10.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa722ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa722ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52682/0 (socket says 172.21.15.37:52682) 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.043+0000 7fa722ffd700 1 -- 172.21.15.37:0/8926313 learned_addr learned my addr 172.21.15.37:0/8926313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa7227fc700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7241019a0 0x7fa724101e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa7237fe700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7241a42c0 0x7fa7241a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 -- 172.21.15.37:0/8926313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7241019a0 msgr2=0x7fa724101e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7241019a0 0x7fa724101e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 -- 172.21.15.37:0/8926313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7241a42c0 msgr2=0x7fa7241a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7241a42c0 0x7fa7241a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 -- 172.21.15.37:0/8926313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7241a6de0 con 0x7fa7241084e0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa722ffd700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa714005b40 tx=0x7fa714004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714026070 con 0x7fa7241084e0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa71400b6f0 con 0x7fa7241084e0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7241a70d0 con 0x7fa7241084e0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714009e60 con 0x7fa7241084e0 2024-04-25T13:18:10.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.044+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7241a7610 con 0x7fa7241084e0 2024-04-25T13:18:10.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.045+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa714029070 con 0x7fa7241084e0 2024-04-25T13:18:10.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.045+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7240640a0 con 0x7fa7241084e0 2024-04-25T13:18:10.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.045+0000 7fa70bfff700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 0x7fa70c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:10.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.046+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa71404d370 con 0x7fa7241084e0 2024-04-25T13:18:10.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.046+0000 7fa7227fc700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 0x7fa70c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:10.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.046+0000 7fa7227fc700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 0x7fa70c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa718009ad0 tx=0x7fa718009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:10.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.055+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa714013070 con 0x7fa7241084e0 2024-04-25T13:18:10.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mon.b calling monitor election 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mon.b calling monitor election 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mon.c calling monitor election 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mon.a calling monitor election 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: monmap epoch 3 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: last_changed 2024-04-25T13:18:02.020944+0000 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: min_mon_release 19 (squid) 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: election_strategy: 1 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:18:10.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: 1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.c 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: fsmap 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: mgrmap e13: x(active, since 63s) 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: overall HEALTH_OK 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:10.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.283+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa7241a7870 con 0x7fa7241084e0 2024-04-25T13:18:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.283+0000 7fa70bfff700 1 -- 172.21.15.37:0/8926313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fa714030120 con 0x7fa7241084e0 2024-04-25T13:18:10.284 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:10.284 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T13:18:10.284 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:10.284 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 msgr2=0x7fa70c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 0x7fa70c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa718009ad0 tx=0x7fa718009210 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 msgr2=0x7fa724101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa714005b40 tx=0x7fa714004f80 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 shutdown_connections 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa70c039ae0 0x7fa70c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7241a42c0 0x7fa7241a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7241084e0 0x7fa724101460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 --2- 172.21.15.37:0/8926313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa7241019a0 0x7fa724101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.285+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 >> 172.21.15.37:0/8926313 conn(0x7fa724076ea0 msgr2=0x7fa7240fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.286+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 shutdown_connections 2024-04-25T13:18:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:10.286+0000 7fa72933c700 1 -- 172.21.15.37:0/8926313 wait complete. 2024-04-25T13:18:10.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:10.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.b calling monitor election 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.b calling monitor election 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.c calling monitor election 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.a calling monitor election 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: monmap epoch 3 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: last_changed 2024-04-25T13:18:02.020944+0000 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: created 2024-04-25T13:15:55.450143+0000 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: min_mon_release 19 (squid) 2024-04-25T13:18:10.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: election_strategy: 1 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 1: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.c 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: 2: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.b 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: fsmap 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: mgrmap e13: x(active, since 63s) 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: overall HEALTH_OK 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:10.392 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: Reconfiguring daemon mon.a on smithi037 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/8926313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: Reconfiguring daemon mon.a on smithi037 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/8926313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: Reconfiguring daemon mon.a on smithi037 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/8926313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:11.488 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T13:18:11.488 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:18:11.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:18:11.516 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:18:11.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:11.582 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:18:11.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:18:11.614 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:18:11.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:11.682 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:18:11.683 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T13:18:11.717 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:18:11.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:18:11.785 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-04-25T13:18:11.785 INFO:tasks.cephadm:Adding mgr.y on smithi044 2024-04-25T13:18:11.785 INFO:tasks.cephadm:Adding mgr.z on smithi069 2024-04-25T13:18:11.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi044=y;smithi069=z' 2024-04-25T13:18:12.060 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:18:13.207 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:13.207 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.207 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.207 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:18:13.207 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:13.208 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:13.208 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:13.208 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:12 smithi069 ceph-mon[32534]: Reconfiguring daemon mon.b on smithi044 2024-04-25T13:18:13.292 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:13.293 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:12 smithi044 ceph-mon[32590]: Reconfiguring daemon mon.b on smithi044 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: Reconfiguring mon.b (monmap changed)... 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:12 smithi037 ceph-mon[29204]: Reconfiguring daemon mon.b on smithi044 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- 172.21.15.69:0/1345737387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d781025d0 msgr2=0x7f5d781029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/1345737387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d781025d0 0x7f5d781029f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009910 tx=0x7f5d6c009c20 comp rx=0 tx=0).stop 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- 172.21.15.69:0/1345737387 shutdown_connections 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/1345737387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d781025d0 0x7f5d781029f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- 172.21.15.69:0/1345737387 >> 172.21.15.69:0/1345737387 conn(0x7f5d780fdb70 msgr2=0x7f5d780fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:13.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- 172.21.15.69:0/1345737387 shutdown_connections 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- 172.21.15.69:0/1345737387 wait complete. 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 Processor -- start 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- start start 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d78074e30 0x7f5d780752b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 0x7f5d781a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78108510 con 0x7f5d780757f0 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78108390 con 0x7f5d78074e30 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.388+0000 7f5d7f21b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78108690 con 0x7f5d781025d0 2024-04-25T13:18:13.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37520/0 (socket says 172.21.15.69:37520) 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 -- 172.21.15.69:0/792985238 learned_addr learned my addr 172.21.15.69:0/792985238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d77fff700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d78074e30 0x7f5d780752b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7d7b8700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 0x7f5d781a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 -- 172.21.15.69:0/792985238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d78074e30 msgr2=0x7f5d780752b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d78074e30 0x7f5d780752b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 -- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 msgr2=0x7f5d781a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 0x7f5d781a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 -- 172.21.15.69:0/792985238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d781a2340 con 0x7f5d781025d0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7d7b8700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 0x7f5d781a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7cfb7700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c005e00 tx=0x7f5d6c005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c027070 con 0x7f5d781025d0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d781a25d0 con 0x7f5d781025d0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d6c00be10 con 0x7f5d781025d0 2024-04-25T13:18:13.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.389+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c022da0 con 0x7f5d781025d0 2024-04-25T13:18:13.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.390+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781a2b80 con 0x7f5d781025d0 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.390+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5d6c02a070 con 0x7f5d781025d0 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.391+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d780640a0 con 0x7f5d781025d0 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.391+0000 7f5d75ffb700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 0x7f5d6803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.391+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d6c04dd60 con 0x7f5d781025d0 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.391+0000 7f5d77fff700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 0x7f5d6803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:13.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.392+0000 7f5d77fff700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 0x7f5d6803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009ad0 tx=0x7f5d64009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:13.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.396+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d6c053020 con 0x7f5d781025d0 2024-04-25T13:18:13.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.601+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi044=y;smithi069=z", "target": ["mon-mgr", ""]}) -- 0x7f5d78062e80 con 0x7f5d68039ae0 2024-04-25T13:18:13.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.608+0000 7f5d75ffb700 1 -- 172.21.15.69:0/792985238 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5d78062e80 con 0x7f5d68039ae0 2024-04-25T13:18:13.607 INFO:teuthology.orchestra.run.smithi069.stdout:Scheduled mgr update... 2024-04-25T13:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 msgr2=0x7f5d6803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:13.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 0x7f5d6803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009ad0 tx=0x7f5d64009210 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 msgr2=0x7f5d780767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c005e00 tx=0x7f5d6c005020 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 shutdown_connections 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5d68039ae0 0x7f5d6803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d780757f0 0x7f5d781a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d781025d0 0x7f5d780767e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 --2- 172.21.15.69:0/792985238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d78074e30 0x7f5d780752b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 >> 172.21.15.69:0/792985238 conn(0x7f5d780fdb70 msgr2=0x7f5d780fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 shutdown_connections 2024-04-25T13:18:13.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:18:13.610+0000 7f5d7f21b700 1 -- 172.21.15.69:0/792985238 wait complete. 2024-04-25T13:18:14.173 DEBUG:teuthology.orchestra.run.smithi044:mgr.y> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T13:18:14.176 DEBUG:teuthology.orchestra.run.smithi069:mgr.z> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T13:18:14.178 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T13:18:14.179 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:18:14.179 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:18:14.198 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:18:14.198 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-25T13:18:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:18:14.257 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 13:17:18.744182870 +0000 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 13:12:14.915449707 +0000 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 13:12:14.915449707 +0000 2024-04-25T13:18:14.258 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 13:12:14.915449707 +0000 2024-04-25T13:18:14.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:18:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T13:18:14.325 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T13:18:14.326 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000228485 s, 2.2 MB/s 2024-04-25T13:18:14.327 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:18:14.385 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-25T13:18:14.443 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 13:17:18.774182464 +0000 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 13:12:15.274444757 +0000 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 13:12:15.274444757 +0000 2024-04-25T13:18:14.444 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 13:12:15.274444757 +0000 2024-04-25T13:18:14.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:18:14.509 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T13:18:14.509 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T13:18:14.509 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000214274 s, 2.4 MB/s 2024-04-25T13:18:14.510 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:18:14.568 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 13:17:18.800182111 +0000 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 13:12:15.625439920 +0000 2024-04-25T13:18:14.625 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 13:12:15.625439920 +0000 2024-04-25T13:18:14.626 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 13:12:15.625439920 +0000 2024-04-25T13:18:14.626 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:18:14.690 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T13:18:14.690 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T13:18:14.690 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000209546 s, 2.4 MB/s 2024-04-25T13:18:14.691 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:18:14.748 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 13:17:18.826181759 +0000 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 13:12:15.977435068 +0000 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 13:12:15.977435068 +0000 2024-04-25T13:18:14.805 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 13:12:15.977435068 +0000 2024-04-25T13:18:14.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:18:14.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi044=y;smithi069=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:14.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: Saving service mgr spec with placement smithi037=x;smithi044=y;smithi069=z;count:3 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: Reconfiguring daemon mon.c on smithi069 2024-04-25T13:18:14.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:14 smithi037 ceph-mon[29204]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:14.869 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T13:18:14.869 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T13:18:14.869 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000198767 s, 2.6 MB/s 2024-04-25T13:18:14.870 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi044=y;smithi069=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: Saving service mgr spec with placement smithi037=x;smithi044=y;smithi069=z;count:3 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: Reconfiguring daemon mon.c on smithi069 2024-04-25T13:18:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:14 smithi069 ceph-mon[32534]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:14.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi044=y;smithi069=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: Saving service mgr spec with placement smithi037=x;smithi044=y;smithi069=z;count:3 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: Reconfiguring mon.c (monmap changed)... 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: Reconfiguring daemon mon.c on smithi069 2024-04-25T13:18:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:14 smithi044 ceph-mon[32590]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:14.927 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-25T13:18:14.927 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:18:14.947 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:18:14.947 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-25T13:18:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:18:15.006 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.006 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T13:18:15.006 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.007 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.007 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-25 13:18:01.460705873 +0000 2024-04-25T13:18:15.007 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-25 13:12:36.939852632 +0000 2024-04-25T13:18:15.007 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-25 13:12:36.939852632 +0000 2024-04-25T13:18:15.007 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-25 13:12:36.939852632 +0000 2024-04-25T13:18:15.007 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:18:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-25T13:18:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-25T13:18:15.074 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000241516 s, 2.1 MB/s 2024-04-25T13:18:15.076 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:18:15.134 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-25T13:18:15.193 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-25 13:18:01.460705873 +0000 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-25 13:12:37.284847135 +0000 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-25 13:12:37.284847135 +0000 2024-04-25T13:18:15.194 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-25 13:12:37.284847135 +0000 2024-04-25T13:18:15.194 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:18:15.261 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-25T13:18:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-25T13:18:15.262 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000207254 s, 2.5 MB/s 2024-04-25T13:18:15.263 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:18:15.322 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-25 13:18:01.461705857 +0000 2024-04-25T13:18:15.379 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-25 13:12:37.631841605 +0000 2024-04-25T13:18:15.380 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-25 13:12:37.631841605 +0000 2024-04-25T13:18:15.380 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-25 13:12:37.631841605 +0000 2024-04-25T13:18:15.380 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:18:15.445 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-25T13:18:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-25T13:18:15.446 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000189829 s, 2.7 MB/s 2024-04-25T13:18:15.447 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:18:15.506 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-25T13:18:15.564 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-25 13:18:01.461705857 +0000 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-25 13:12:37.984835980 +0000 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-25 13:12:37.984835980 +0000 2024-04-25T13:18:15.565 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-25 13:12:37.984835980 +0000 2024-04-25T13:18:15.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:18:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-25T13:18:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-25T13:18:15.631 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000205909 s, 2.5 MB/s 2024-04-25T13:18:15.632 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:18:15.691 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-04-25T13:18:15.691 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:18:15.709 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:18:15.709 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-04-25T13:18:15.766 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:18:15.766 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.766 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 13:17:51.898555206 +0000 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 13:12:25.811011466 +0000 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 13:12:25.811011466 +0000 2024-04-25T13:18:15.767 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 13:12:25.811011466 +0000 2024-04-25T13:18:15.767 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:18:15.830 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T13:18:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T13:18:15.831 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000259369 s, 2.0 MB/s 2024-04-25T13:18:15.832 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:18:15.888 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 13:17:51.899555185 +0000 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 13:12:26.156004052 +0000 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 13:12:26.156004052 +0000 2024-04-25T13:18:15.943 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 13:12:26.156004052 +0000 2024-04-25T13:18:15.944 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:18:16.008 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T13:18:16.008 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T13:18:16.008 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000185155 s, 2.8 MB/s 2024-04-25T13:18:16.009 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:18:16.065 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 13:17:51.899555185 +0000 2024-04-25T13:18:16.121 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 13:12:26.513996331 +0000 2024-04-25T13:18:16.122 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 13:12:26.513996331 +0000 2024-04-25T13:18:16.122 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 13:12:26.513996331 +0000 2024-04-25T13:18:16.122 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:18:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T13:18:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T13:18:16.185 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.00018457 s, 2.8 MB/s 2024-04-25T13:18:16.186 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:18:16.244 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T13:18:16.299 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-04-25 13:17:51.899555185 +0000 2024-04-25T13:18:16.300 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-04-25 13:12:26.848989104 +0000 2024-04-25T13:18:16.300 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-04-25 13:12:26.848989104 +0000 2024-04-25T13:18:16.300 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-04-25 13:12:26.848989104 +0000 2024-04-25T13:18:16.300 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:18:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-04-25T13:18:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-04-25T13:18:16.362 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000210577 s, 2.4 MB/s 2024-04-25T13:18:16.363 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:16.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.400 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-25T13:18:16.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:18:16.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:16.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:17 smithi069 ceph-mon[32534]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:17 smithi069 ceph-mon[32534]: Deploying daemon mgr.z on smithi069 2024-04-25T13:18:17.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:17 smithi044 ceph-mon[32590]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:17.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:17 smithi044 ceph-mon[32590]: Deploying daemon mgr.z on smithi069 2024-04-25T13:18:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:17 smithi037 ceph-mon[29204]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:17 smithi037 ceph-mon[29204]: Deploying daemon mgr.z on smithi069 2024-04-25T13:18:18.359 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:18 smithi069 systemd[1]: Started Ceph mgr.z for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112955 s, 928 MB/s 2024-04-25T13:18:18.559 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T13:18:18.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:18.358+0000 7f037180e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:18:18.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:18.547+0000 7f037180e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:18:18.955 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:18.649+0000 7f037180e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:18:18.955 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:18.954+0000 7f037180e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:18:19.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-25T13:18:19.375 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:19.038+0000 7f037180e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:18:19.466 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:18:19.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:19.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:19.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:19 smithi037 ceph-mon[29204]: Deploying daemon mgr.y on smithi044 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:19.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:19 smithi069 ceph-mon[32534]: Deploying daemon mgr.y on smithi044 2024-04-25T13:18:19.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:19.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:19.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:19.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T13:18:19.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:19.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:19.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:19 smithi044 ceph-mon[32590]: Deploying daemon mgr.y on smithi044 2024-04-25T13:18:20.547 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:20.211+0000 7f037180e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:18:20.547 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:20.380+0000 7f037180e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:18:20.547 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:20.546+0000 7f037180e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:18:20.914 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:20 smithi044 ceph-mgr[33990]: -- 172.21.15.44:0/3224520921 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d69a2e5a0 con 0x556d69a02800 2024-04-25T13:18:21.126 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:20.829+0000 7f037180e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:18:21.126 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:20.921+0000 7f037180e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:18:21.126 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:21.126+0000 7f037180e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 -- 172.21.15.37:0/382665434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36981087b0 msgr2=0x7f369810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 --2- 172.21.15.37:0/382665434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36981087b0 0x7f369810ac20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f369000b0a0 tx=0x7f369000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 -- 172.21.15.37:0/382665434 shutdown_connections 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 --2- 172.21.15.37:0/382665434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36981087b0 0x7f369810ac20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 --2- 172.21.15.37:0/382665434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698103550 0x7f3698105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 --2- 172.21.15.37:0/382665434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3698105e80 0x7f3698108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 -- 172.21.15.37:0/382665434 >> 172.21.15.37:0/382665434 conn(0x7f36980fcf60 msgr2=0x7f36980ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 -- 172.21.15.37:0/382665434 shutdown_connections 2024-04-25T13:18:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.136+0000 7f369e434700 1 -- 172.21.15.37:0/382665434 wait complete. 2024-04-25T13:18:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.137+0000 7f369e434700 1 Processor -- start 2024-04-25T13:18:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.137+0000 7f369e434700 1 -- start start 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.137+0000 7f369e434700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.137+0000 7f369e434700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698105e80 0x7f36981990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.137+0000 7f369e434700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36981087b0 0x7f36981a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369e434700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f369810e570 con 0x7f3698103550 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369e434700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f369810e3f0 con 0x7f36981087b0 2024-04-25T13:18:21.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369e434700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f369810e6f0 con 0x7f3698105e80 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38208/0 (socket says 172.21.15.37:38208) 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 -- 172.21.15.37:0/1658814326 learned_addr learned my addr 172.21.15.37:0/1658814326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f36977fe700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698105e80 0x7f36981990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369c9d1700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36981087b0 0x7f36981a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 -- 172.21.15.37:0/1658814326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698105e80 msgr2=0x7f36981990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698105e80 0x7f36981990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 -- 172.21.15.37:0/1658814326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36981087b0 msgr2=0x7f36981a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36981087b0 0x7f36981a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 -- 172.21.15.37:0/1658814326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a07e0 con 0x7f3698103550 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f3697fff700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9d0 tx=0x7f368800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368800cbd0 con 0x7f3698103550 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f368800ee70 con 0x7f3698103550 2024-04-25T13:18:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a0ad0 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688018680 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.138+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a1060 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.139+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3688010430 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.139+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698069730 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.139+0000 7f36957fa700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 0x7f368003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.140+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f368804cb70 con 0x7f3698103550 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.140+0000 7f36977fe700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 0x7f368003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.140+0000 7f36977fe700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 0x7f368003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f368c007be0 tx=0x7f368c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.145+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3688018d40 con 0x7f3698103550 2024-04-25T13:18:21.198 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:20 smithi044 bash[33971]: 554bc2fe87fc8bbf90a535002db03879b458b96d9e49fcb77f7554656607435e 2024-04-25T13:18:21.198 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:20 smithi044 systemd[1]: Started Ceph mgr.y for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:18:21.199 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:21.005+0000 7f9db9ab6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:18:21.199 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:21.198+0000 7f9db9ab6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:18:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:21.364+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3698062e80 con 0x7f3680039890 2024-04-25T13:18:21.378 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:21.324+0000 7f037180e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:21.634 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:21.638 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:21.308+0000 7f9db9ab6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:21.889 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:21.638+0000 7f9db9ab6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:18:21.889 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:21.740+0000 7f9db9ab6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:18:22.381 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:21.951+0000 7f037180e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:18:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:22 smithi037 ceph-mon[29204]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:22.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:22 smithi069 ceph-mon[32534]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:22.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:22.386+0000 7f037180e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:18:22.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:22.482+0000 7f037180e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:18:22.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:22 smithi044 ceph-mon[32590]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:23.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:22.682+0000 7f037180e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:18:23.381 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.027+0000 7f9db9ab6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:18:23.381 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.205+0000 7f9db9ab6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:18:23.381 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.380+0000 7f9db9ab6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.207+0000 7f037180e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.298+0000 7f037180e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.539 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:23.540 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:23.540 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:23.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:23.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:23.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T13:18:23.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T13:18:23.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:23.875 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.538+0000 7f037180e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:18:23.876 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.668+0000 7f037180e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:18:23.876 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.761+0000 7f037180e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:18:23.876 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:23.875+0000 7f037180e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:18:23.983 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.679+0000 7f9db9ab6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:18:23.984 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.774+0000 7f9db9ab6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:18:23.984 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:23.983+0000 7f9db9ab6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:18:24.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:24.183+0000 7f9db9ab6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:18:24.429 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:24.171+0000 7f037180e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2653051109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]': finished 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:18:24.768 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2653051109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]': finished 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:18:24.808 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2653051109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]: dispatch 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f78d67e4-010a-4965-86af-acd038f6cdf1"}]': finished 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T13:18:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:25.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:24.767+0000 7f037180e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:18:25.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:18:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:18:24.980+0000 7f037180e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:18:25.139 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:24.808+0000 7f9db9ab6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:18:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:25.444+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f3688018820 con 0x7f3698103550 2024-04-25T13:18:25.551 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:25.244+0000 7f9db9ab6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:18:25.551 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:25.343+0000 7f9db9ab6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:18:25.551 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:25.551+0000 7f9db9ab6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1848929517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: Standby manager daemon z started 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:18:25.552 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:25 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1848929517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: Standby manager daemon z started 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:18:25.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:25 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1848929517' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: Standby manager daemon z started 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:18:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:25 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/1493248904' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:26.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.118+0000 7f9db9ab6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:18:26.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.212+0000 7f9db9ab6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:18:26.702 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-mon[32590]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T13:18:26.703 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:18:26.703 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:26.703 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.465+0000 7f9db9ab6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:18:26.703 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.603+0000 7f9db9ab6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:18:26.703 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.702+0000 7f9db9ab6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:18:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:26 smithi037 ceph-mon[29204]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T13:18:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:18:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:26 smithi069 ceph-mon[32534]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-25T13:18:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:18:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:27.139 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:26.822+0000 7f9db9ab6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:18:27.450 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:27.159+0000 7f9db9ab6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:18:27.787 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:27 smithi044 ceph-mon[32590]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:27.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:27 smithi037 ceph-mon[29204]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:27.884 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:27 smithi069 ceph-mon[32534]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:28.139 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:27.787+0000 7f9db9ab6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:18:28.139 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:18:28.011+0000 7f9db9ab6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:18:28.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:28.463+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f3688010720 con 0x7f3698103550 2024-04-25T13:18:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:28 smithi037 ceph-mon[29204]: Standby manager daemon y started 2024-04-25T13:18:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:28 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:18:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:28 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:28 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:18:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:28 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:28 smithi069 ceph-mon[32534]: Standby manager daemon y started 2024-04-25T13:18:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:28 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:18:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:28 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:28 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:18:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:28 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-mon[32590]: Standby manager daemon y started 2024-04-25T13:18:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:18:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:18:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:18:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:28 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/3990788639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:18:29.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:29 smithi037 ceph-mon[29204]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:29.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:29 smithi037 ceph-mon[29204]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T13:18:29.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:18:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:29 smithi069 ceph-mon[32534]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:29 smithi069 ceph-mon[32534]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T13:18:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:18:29.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:29 smithi044 ceph-mon[32590]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:29.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:29 smithi044 ceph-mon[32590]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-04-25T13:18:29.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:18:31.579 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:31.579 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:31.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:31.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:31.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:31.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:31.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:31.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:31.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:31.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:33 smithi037 ceph-mon[29204]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:18:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:33 smithi069 ceph-mon[32534]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:18:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:33.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:33 smithi044 ceph-mon[32590]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:33.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T13:18:33.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:34.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:34 smithi037 ceph-mon[29204]: Deploying daemon osd.0 on smithi037 2024-04-25T13:18:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:34 smithi069 ceph-mon[32534]: Deploying daemon osd.0 on smithi037 2024-04-25T13:18:34.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:34 smithi044 ceph-mon[32590]: Deploying daemon osd.0 on smithi037 2024-04-25T13:18:35.546 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:35 smithi037 ceph-mon[29204]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:35 smithi069 ceph-mon[32534]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:35 smithi044 ceph-mon[32590]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:37 smithi037 ceph-mon[29204]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:37.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:37 smithi069 ceph-mon[32534]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:37.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:37 smithi044 ceph-mon[32590]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:39 smithi037 ceph-mon[29204]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:39 smithi069 ceph-mon[32534]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:39 smithi044 ceph-mon[32590]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:39.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:41.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:41.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:41.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:41.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:41.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:41.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:41.924 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-04-25T13:18:41.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.923+0000 7f36957fa700 1 -- 172.21.15.37:0/1658814326 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3698062e80 con 0x7f3680039890 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.925+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 msgr2=0x7f368003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.925+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 0x7f368003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f368c007be0 tx=0x7f368c0073d0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.925+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 msgr2=0x7f3698198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.925+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f368800e9d0 tx=0x7f368800ece0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 shutdown_connections 2024-04-25T13:18:41.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3680039890 0x7f368003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36981087b0 0x7f36981a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3698103550 0x7f3698198b80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 --2- 172.21.15.37:0/1658814326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3698105e80 0x7f36981990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:41.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 >> 172.21.15.37:0/1658814326 conn(0x7f36980fcf60 msgr2=0x7f3698103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:41.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 shutdown_connections 2024-04-25T13:18:41.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:18:41.926+0000 7f369e434700 1 -- 172.21.15.37:0/1658814326 wait complete. 2024-04-25T13:18:42.574 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T13:18:42.577 INFO:tasks.cephadm:Deploying osd.1 on smithi044 with /dev/vg_nvme/lv_4... 2024-04-25T13:18:42.577 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:18:42.829 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:18:43.071 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:42 smithi037 ceph-mon[29204]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:43.071 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:42 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.071 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:42 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.071 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:42 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:43.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:42 smithi037 ceph-mon[29204]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:18:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:42 smithi069 ceph-mon[32534]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:42 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:42 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:42 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:42 smithi069 ceph-mon[32534]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:18:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:42 smithi044 ceph-mon[32590]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:42 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:42 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:42 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:18:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:42 smithi044 ceph-mon[32590]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T13:18:44.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:43 smithi037 ceph-mon[29204]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:18:44.031 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:43 smithi037 ceph-mon[29204]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:18:44.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:43 smithi037 ceph-mon[29204]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:18:44.032 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:43 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:44.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:43 smithi044 ceph-mon[32590]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:18:44.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:43 smithi044 ceph-mon[32590]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:18:44.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:43 smithi044 ceph-mon[32590]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:18:44.052 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:43 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:44.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:43 smithi069 ceph-mon[32534]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T13:18:44.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:43 smithi069 ceph-mon[32534]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T13:18:44.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:43 smithi069 ceph-mon[32534]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:18:44.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:43 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136112 s, 770 MB/s 2024-04-25T13:18:44.710 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-mon[29204]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' 2024-04-25T13:18:45.099 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0[44049]: 2024-04-25T13:18:44.828+0000 7fc1ad024700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:18:45.099 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:18:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0[44049]: 2024-04-25T13:18:44.888+0000 7fc1a460e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.130 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:44 smithi044 ceph-mon[32590]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' 2024-04-25T13:18:45.320 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:44 smithi069 ceph-mon[32534]: from='osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087]' entity='osd.0' 2024-04-25T13:18:45.567 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.263 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:18:46.263 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:18:46.263 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:46.263 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:46.264 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.286 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:18:46.286 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:18:46.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:18:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 -- 172.21.15.44:0/1123439218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c1019e0 msgr2=0x7f585c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 --2- 172.21.15.44:0/1123439218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c1019e0 0x7f585c101e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f58580099c0 tx=0x7f5858009cd0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 -- 172.21.15.44:0/1123439218 shutdown_connections 2024-04-25T13:18:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 --2- 172.21.15.44:0/1123439218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c1019e0 0x7f585c101e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 --2- 172.21.15.44:0/1123439218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 --2- 172.21.15.44:0/1123439218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c102be0 0x7f585c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.106+0000 7f5863c0f700 1 -- 172.21.15.44:0/1123439218 >> 172.21.15.44:0/1123439218 conn(0x7f585c0fcfa0 msgr2=0x7f585c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- 172.21.15.44:0/1123439218 shutdown_connections 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- 172.21.15.44:0/1123439218 wait complete. 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 Processor -- start 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- start start 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c1019e0 0x7f585c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10d470 con 0x7f585c1019e0 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10d2f0 con 0x7f585c102be0 2024-04-25T13:18:47.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.107+0000 7f5863c0f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10d5f0 con 0x7f585c103580 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49218/0 (socket says 172.21.15.44:49218) 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 -- 172.21.15.44:0/2086855100 learned_addr learned my addr 172.21.15.44:0/2086855100 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58621ac700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58619ab700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c1019e0 0x7f585c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 -- 172.21.15.44:0/2086855100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 msgr2=0x7f585c1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 -- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c1019e0 msgr2=0x7f585c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c1019e0 0x7f585c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 -- 172.21.15.44:0/2086855100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c1a07a0 con 0x7f585c102be0 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58621ac700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:18:47.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f58611aa700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c9b0 tx=0x7f584c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c007b00 con 0x7f585c102be0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c007ca0 con 0x7f585c102be0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c018660 con 0x7f585c102be0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c1a0a90 con 0x7f585c102be0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.108+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c1a1020 con 0x7f585c102be0 2024-04-25T13:18:47.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.109+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c069730 con 0x7f585c102be0 2024-04-25T13:18:47.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.110+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f584c00f450 con 0x7f585c102be0 2024-04-25T13:18:47.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.111+0000 7f5852ffd700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 0x7f58480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:18:47.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.111+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f584c0cead0 con 0x7f585c102be0 2024-04-25T13:18:47.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.111+0000 7f58619ab700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 0x7f58480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:18:47.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.112+0000 7f58619ab700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 0x7f58480a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58580036f0 tx=0x7f5858005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:18:47.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.115+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f584c014030 con 0x7f585c102be0 2024-04-25T13:18:47.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:18:47.333+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f585c0ff0f0 con 0x7f58480a3c30 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi037 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] boot 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:18:47.335 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:46 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi037 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] boot 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:18:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:46 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T13:18:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi037 2024-04-25T13:18:47.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T13:18:47.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:47.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: osd.0 [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] boot 2024-04-25T13:18:47.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T13:18:47.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:46 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:18:48.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:47 smithi044 ceph-mon[32590]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:18:48.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:47 smithi044 ceph-mon[32590]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:48.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:48.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:48.170 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:47 smithi037 ceph-mon[29204]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:18:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:47 smithi037 ceph-mon[29204]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:48.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:47 smithi069 ceph-mon[32534]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T13:18:48.477 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:47 smithi069 ceph-mon[32534]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:18:48.478 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:18:48.478 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:18:48.478 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:49.085 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:48 smithi044 ceph-mon[32590]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:49.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:48 smithi037 ceph-mon[29204]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:49.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:48 smithi069 ceph-mon[32534]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2622435694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]': finished 2024-04-25T13:18:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:18:51.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:18:51.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:50 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/165396279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2622435694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]': finished 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:18:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:50 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/165396279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2622435694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]: dispatch 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "037053d7-433f-4d1a-9e7a-d44599f4a488"}]': finished 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:18:51.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:50 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/165396279' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:18:53.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:52 smithi037 ceph-mon[29204]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:52 smithi069 ceph-mon[32534]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:53.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:52 smithi044 ceph-mon[32590]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:55.216 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:54 smithi044 ceph-mon[32590]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:54 smithi037 ceph-mon[29204]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:55.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:54 smithi069 ceph-mon[32534]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:57.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:56 smithi044 ceph-mon[32590]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:57.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:56 smithi037 ceph-mon[29204]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:56 smithi069 ceph-mon[32534]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:58.221 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:18:58.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:58.222 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:57 smithi044 ceph-mon[32590]: Deploying daemon osd.1 on smithi044 2024-04-25T13:18:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:18:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:57 smithi037 ceph-mon[29204]: Deploying daemon osd.1 on smithi044 2024-04-25T13:18:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T13:18:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:18:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:57 smithi069 ceph-mon[32534]: Deploying daemon osd.1 on smithi044 2024-04-25T13:18:59.261 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:18:58 smithi044 ceph-mon[32590]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:18:58 smithi037 ceph-mon[29204]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:18:59.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:18:58 smithi069 ceph-mon[32534]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:00 smithi044 ceph-mon[32590]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:01.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:00 smithi037 ceph-mon[29204]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:00 smithi069 ceph-mon[32534]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:03.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:02 smithi044 ceph-mon[32590]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:03.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:02 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:02 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:02 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:02 smithi037 ceph-mon[29204]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:02 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:02 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:02 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:02 smithi069 ceph-mon[32534]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:02 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:02 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:02 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:05.128 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:04 smithi044 ceph-mon[32590]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:05.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:04 smithi037 ceph-mon[29204]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:04 smithi069 ceph-mon[32534]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:06.053 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 1 on host 'smithi044' 2024-04-25T13:19:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.053+0000 7f5852ffd700 1 -- 172.21.15.44:0/2086855100 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f585c0ff0f0 con 0x7f58480a3c30 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 msgr2=0x7f58480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 0x7f58480a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58580036f0 tx=0x7f5858005fd0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 msgr2=0x7f585c199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c9b0 tx=0x7f584c00ccc0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 shutdown_connections 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f58480a3c30 0x7f58480a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f585c103580 0x7f585c1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f585c1019e0 0x7f585c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 --2- 172.21.15.44:0/2086855100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f585c102be0 0x7f585c199080 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.055+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 >> 172.21.15.44:0/2086855100 conn(0x7f585c0fcfa0 msgr2=0x7f585c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.056+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 shutdown_connections 2024-04-25T13:19:06.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:19:06.056+0000 7f5863c0f700 1 -- 172.21.15.44:0/2086855100 wait complete. 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:06.688 DEBUG:teuthology.orchestra.run.smithi044:osd.1> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T13:19:06.690 INFO:tasks.cephadm:Deploying osd.2 on smithi069 with /dev/vg_nvme/lv_4... 2024-04-25T13:19:06.690 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T13:19:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:19:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:07 smithi069 ceph-mon[32534]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:07 smithi069 ceph-mon[32534]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:07 smithi069 ceph-mon[32534]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:07.718 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:07 smithi044 ceph-mon[32590]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:07.718 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:07 smithi044 ceph-mon[32590]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:07.718 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:07 smithi044 ceph-mon[32590]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:07 smithi037 ceph-mon[29204]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:07 smithi037 ceph-mon[29204]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:07 smithi037 ceph-mon[29204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T13:19:08.767 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T13:19:08.767 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T13:19:08.768 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T13:19:08.768 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T13:19:08.768 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112419 s, 933 MB/s 2024-04-25T13:19:08.768 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T13:19:08.790 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:08 smithi069 ceph-mon[32534]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:19:08.790 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:08 smithi069 ceph-mon[32534]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:19:08.791 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:08.791 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:08 smithi069 ceph-mon[32534]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:08.791 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:08 smithi069 ceph-mon[32534]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:08.827 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:08 smithi044 ceph-mon[32590]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:19:08.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:08 smithi044 ceph-mon[32590]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:19:08.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:08.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:08 smithi044 ceph-mon[32590]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:08.828 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:08 smithi044 ceph-mon[32590]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:08 smithi037 ceph-mon[29204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T13:19:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:08 smithi037 ceph-mon[29204]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T13:19:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:08 smithi037 ceph-mon[29204]: from='osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:08 smithi037 ceph-mon[29204]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:19:09.365 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-04-25T13:19:09.433 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1[37184]: 2024-04-25T13:19:09.405+0000 7f50f3c70700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:19:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='osd.1 ' entity='osd.1' 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='osd.1 ' entity='osd.1' 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.889 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1[37184]: 2024-04-25T13:19:09.551+0000 7f50eba5b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='osd.1 ' entity='osd.1' 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:09.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:09.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:09.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:09.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:10.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:10 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi044 2024-04-25T13:19:10.801 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:10 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-25T13:19:10.801 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:10 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi044 2024-04-25T13:19:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:10 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-25T13:19:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:10 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi044 2024-04-25T13:19:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:10 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 17093M 2024-04-25T13:19:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- 172.21.15.69:0/1991312634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4103560 msgr2=0x7f7da4105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/1991312634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4103560 0x7f7da4105950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009990 tx=0x7f7d94009ca0 comp rx=0 tx=0).stop 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- 172.21.15.69:0/1991312634 shutdown_connections 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/1991312634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da410ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/1991312634 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4103560 0x7f7da4105950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:10.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/1991312634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4105e90 0x7f7da4108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- 172.21.15.69:0/1991312634 >> 172.21.15.69:0/1991312634 conn(0x7f7da40fcf90 msgr2=0x7f7da40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- 172.21.15.69:0/1991312634 shutdown_connections 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- 172.21.15.69:0/1991312634 wait complete. 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 Processor -- start 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.996+0000 7f7dabbb5700 1 -- start start 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 0x7f7da4102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da4101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410e5d0 con 0x7f7da4103560 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410e450 con 0x7f7da41087c0 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7da410e750 con 0x7f7da4105e90 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45830/0 (socket says 172.21.15.69:45830) 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 -- 172.21.15.69:0/199612328 learned_addr learned my addr 172.21.15.69:0/199612328 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:19:10.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9951700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 0x7f7da4102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7daa152700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da4101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 -- 172.21.15.69:0/199612328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 msgr2=0x7f7da4101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da4101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 -- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 msgr2=0x7f7da4102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 0x7f7da4102e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 -- 172.21.15.69:0/199612328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da4102290 con 0x7f7da4105e90 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7daa152700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9951700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 0x7f7da4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7da9150700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7da000d950 tx=0x7f7da000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0016070 con 0x7f7da4105e90 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41a2d10 con 0x7f7da4105e90 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da0010460 con 0x7f7da4105e90 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.997+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da000f620 con 0x7f7da4105e90 2024-04-25T13:19:11.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.998+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a3270 con 0x7f7da4105e90 2024-04-25T13:19:11.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:10.999+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4069730 con 0x7f7da4105e90 2024-04-25T13:19:11.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.000+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f7da0019070 con 0x7f7da4105e90 2024-04-25T13:19:11.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.000+0000 7f7d9affd700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 0x7f7d900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:11.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.000+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f7da00ce800 con 0x7f7da4105e90 2024-04-25T13:19:11.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.000+0000 7f7da9951700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 0x7f7d900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:11.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.001+0000 7f7da9951700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 0x7f7d900a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b5c0 tx=0x7f7d94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:11.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.004+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da009ca30 con 0x7f7da4105e90 2024-04-25T13:19:11.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:11.213+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7da4062e80 con 0x7f7d900a3bb0 2024-04-25T13:19:11.527 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:11.527 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] boot 2024-04-25T13:19:11.527 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:19:11.527 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:11.528 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:11.528 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:11.528 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:11.528 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] boot 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:11.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: osd.1 [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] boot 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:11.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:12.724 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:12 smithi069 ceph-mon[32534]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:19:12.724 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:12 smithi069 ceph-mon[32534]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:12 smithi037 ceph-mon[29204]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:19:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:12 smithi037 ceph-mon[29204]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:12 smithi044 ceph-mon[32590]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T13:19:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:12 smithi044 ceph-mon[32590]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/2674081758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]': finished 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:14 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/1911319899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/2674081758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]': finished 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:14 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/1911319899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/2674081758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]: dispatch 2024-04-25T13:19:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3317a25e-0c74-403e-94a3-fba65b8e6733"}]': finished 2024-04-25T13:19:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T13:19:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:14 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/1911319899' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:16 smithi037 ceph-mon[29204]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:16 smithi069 ceph-mon[32534]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:17.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:16 smithi044 ceph-mon[32590]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:19.076 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:18 smithi069 ceph-mon[32534]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:18 smithi037 ceph-mon[29204]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:19.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:18 smithi044 ceph-mon[32590]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:21.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:20 smithi037 ceph-mon[29204]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:20 smithi069 ceph-mon[32534]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:21.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:20 smithi044 ceph-mon[32590]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:19:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:21 smithi037 ceph-mon[29204]: Deploying daemon osd.2 on smithi069 2024-04-25T13:19:22.105 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:19:22.105 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:22.105 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:21 smithi069 ceph-mon[32534]: Deploying daemon osd.2 on smithi069 2024-04-25T13:19:22.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T13:19:22.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:22.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:21 smithi044 ceph-mon[32590]: Deploying daemon osd.2 on smithi069 2024-04-25T13:19:23.043 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:22 smithi069 ceph-mon[32534]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:22 smithi037 ceph-mon[29204]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:23.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:22 smithi044 ceph-mon[32590]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:25.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:24 smithi037 ceph-mon[29204]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:24 smithi069 ceph-mon[32534]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:25.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:24 smithi044 ceph-mon[32590]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:27.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:27 smithi069 ceph-mon[32534]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:27.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:27 smithi037 ceph-mon[29204]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:27.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:27 smithi044 ceph-mon[32590]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:27.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:29.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:29.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:29.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:29.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:30.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.232+0000 7f7d9affd700 1 -- 172.21.15.69:0/199612328 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7da4062e80 con 0x7f7d900a3bb0 2024-04-25T13:19:30.234 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 2 on host 'smithi069' 2024-04-25T13:19:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 msgr2=0x7f7d900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 0x7f7d900a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400b5c0 tx=0x7f7d94005fd0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 msgr2=0x7f7da4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7da000d950 tx=0x7f7da000dd10 comp rx=0 tx=0).stop 2024-04-25T13:19:30.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 shutdown_connections 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d900a3bb0 0x7f7d900a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7da41087c0 0x7f7da4101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da4103560 0x7f7da4102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 --2- 172.21.15.69:0/199612328 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7da4105e90 0x7f7da4101510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 >> 172.21.15.69:0/199612328 conn(0x7f7da40fcf90 msgr2=0x7f7da4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 shutdown_connections 2024-04-25T13:19:30.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:19:30.235+0000 7f7dabbb5700 1 -- 172.21.15.69:0/199612328 wait complete. 2024-04-25T13:19:30.816 DEBUG:teuthology.orchestra.run.smithi069:osd.2> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T13:19:30.818 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-25T13:19:30.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:19:31.073 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:31 smithi069 ceph-mon[32534]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:31.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:31 smithi037 ceph-mon[29204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:31.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:31.642 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.642 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:31.642 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:31.642 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:31.642 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:31 smithi044 ceph-mon[32590]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T13:19:32.353 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:32 smithi069 ceph-mon[32534]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:19:32.353 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:32 smithi069 ceph-mon[32534]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:19:32.353 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:32.353 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:32 smithi069 ceph-mon[32534]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:32.353 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:32 smithi069 ceph-mon[32534]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:32.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:32 smithi037 ceph-mon[29204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:19:32.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:32 smithi037 ceph-mon[29204]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:19:32.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:32.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:32 smithi037 ceph-mon[29204]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:32.518 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:32 smithi037 ceph-mon[29204]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:32.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:32 smithi044 ceph-mon[32590]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T13:19:32.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:32 smithi044 ceph-mon[32590]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T13:19:32.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:32.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:32 smithi044 ceph-mon[32590]: from='osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:32.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:32 smithi044 ceph-mon[32590]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:19:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:19:33.099 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:19:33.100 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T13:19:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T13:19:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112635 s, 931 MB/s 2024-04-25T13:19:33.100 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T13:19:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:19:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:19:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-mon[32534]: from='osd.2 ' entity='osd.2' 2024-04-25T13:19:33.386 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2[37663]: 2024-04-25T13:19:33.156+0000 7f54efb4e700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:19:33.386 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2[37663]: 2024-04-25T13:19:33.306+0000 7f54e7939700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:19:33.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:33.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:19:33.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:19:33.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:33 smithi037 ceph-mon[29204]: from='osd.2 ' entity='osd.2' 2024-04-25T13:19:33.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:33.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:19:33.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T13:19:33.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:33.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:33 smithi044 ceph-mon[32590]: from='osd.2 ' entity='osd.2' 2024-04-25T13:19:33.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-25T13:19:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:34.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:34.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:34.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:34 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:34.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:34.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:34 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:34.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:19:34.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:34 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi069 2024-04-25T13:19:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T13:19:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] boot 2024-04-25T13:19:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:19:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:35 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi069 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] boot 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:19:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:35 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi069 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 17088M 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: osd.2 [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] boot 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T13:19:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:35 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.668+0000 7f9fd0e31700 1 -- 172.21.15.37:0/1807835309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc108810 msgr2=0x7f9fcc10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.668+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/1807835309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc108810 0x7f9fcc10ac20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400b0a0 tx=0x7f9fc400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 -- 172.21.15.37:0/1807835309 shutdown_connections 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/1807835309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc108810 0x7f9fcc10ac20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/1807835309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc1035b0 0x7f9fcc1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/1807835309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 -- 172.21.15.37:0/1807835309 >> 172.21.15.37:0/1807835309 conn(0x7f9fcc0fcf60 msgr2=0x7f9fcc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 -- 172.21.15.37:0/1807835309 shutdown_connections 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 -- 172.21.15.37:0/1807835309 wait complete. 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 Processor -- start 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.669+0000 7f9fd0e31700 1 -- start start 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc1035b0 0x7f9fcc198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc108810 0x7f9fcc1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10e540 con 0x7f9fcc105ee0 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10e3c0 con 0x7f9fcc1035b0 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fd0e31700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10e6c0 con 0x7f9fcc108810 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:35.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59202/0 (socket says 172.21.15.37:59202) 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 -- 172.21.15.37:0/3337249602 learned_addr learned my addr 172.21.15.37:0/3337249602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fca59c700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc1035b0 0x7f9fcc198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fcad9d700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc108810 0x7f9fcc1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 -- 172.21.15.37:0/3337249602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc108810 msgr2=0x7f9fcc1a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc108810 0x7f9fcc1a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 -- 172.21.15.37:0/3337249602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc1035b0 msgr2=0x7f9fcc198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc1035b0 0x7f9fcc198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:35.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 -- 172.21.15.37:0/3337249602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc1a07f0 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.670+0000 7f9fc9d9b700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000d980 tx=0x7f9fc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.671+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0016070 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.671+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc0009d80 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.671+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc1a0ae0 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.671+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000f3c0 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.671+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1a1070 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.672+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc069730 con 0x7f9fcc105ee0 2024-04-25T13:19:35.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.673+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f9fc0019070 con 0x7f9fcc105ee0 2024-04-25T13:19:35.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.673+0000 7f9fbb7fe700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 0x7f9fb40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:19:35.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.674+0000 7f9fca59c700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 0x7f9fb40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:19:35.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.674+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f9fc00ceb60 con 0x7f9fcc105ee0 2024-04-25T13:19:35.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.674+0000 7f9fca59c700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 0x7f9fb40a6170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005950 tx=0x7f9fbc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:19:35.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.677+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fc009cad0 con 0x7f9fcc105ee0 2024-04-25T13:19:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:35.897+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9fcc062e80 con 0x7f9fb40a3cb0 2024-04-25T13:19:36.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:36 smithi037 ceph-mon[29204]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:19:36.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:36.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:36.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:36.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:36 smithi069 ceph-mon[32534]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:19:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:36 smithi044 ceph-mon[32590]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T13:19:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:19:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:19:36.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:37.509 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 ceph-mon[32534]: pgmap v84: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:37.510 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 ceph-mon[32534]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.510 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:37.510 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 ceph-mon[32534]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:19:37.510 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:37.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 ceph-mon[29204]: pgmap v84: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:37.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 ceph-mon[29204]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:37.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 ceph-mon[29204]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:19:37.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48907]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48907]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48910]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.531 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48910]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 ceph-mon[32590]: pgmap v84: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:37.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 ceph-mon[32590]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:19:37.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:37.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 ceph-mon[32590]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T13:19:37.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41298]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41298]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41298]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41298]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41301]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:37.641 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41301]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.642 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41301]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.642 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41301]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.796 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:37.796 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48934]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48934]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48937]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.797 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:37 smithi037 sudo[48937]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.885 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:37.885 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.885 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.885 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41722]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.885 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41725]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:37.886 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41725]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.886 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41725]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.886 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41725]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41728]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41728]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41728]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41728]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41731]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41731]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:37.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:37 smithi069 sudo[41731]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:38.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41304]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41304]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41304]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41307]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T13:19:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41307]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T13:19:38.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:37 smithi044 sudo[41307]: pam_unix(sudo:session): session closed for user root 2024-04-25T13:19:38.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:38.573+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9fc0098a40 con 0x7f9fcc105ee0 2024-04-25T13:19:38.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.575 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.576 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:38 smithi037 ceph-mon[29204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:19:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.637 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.637 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.637 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:38 smithi069 ceph-mon[32534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:19:38.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:19:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:38 smithi044 ceph-mon[32590]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]: dispatch 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]': finished 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:19:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]: dispatch 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]': finished 2024-04-25T13:19:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:19:39.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T13:19:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T13:19:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]: dispatch 2024-04-25T13:19:39.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/829759512' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "97436b7d-2b40-4fbb-b523-4773cbf165f1"}]': finished 2024-04-25T13:19:39.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T13:19:39.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:40.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:40 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3517614573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:40.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:40 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3517614573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:40.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:40 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3517614573' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:19:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:41 smithi037 ceph-mon[29204]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:41 smithi069 ceph-mon[32534]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:41 smithi044 ceph-mon[32590]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:43 smithi037 ceph-mon[29204]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:43.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:43 smithi069 ceph-mon[32534]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:43.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:43 smithi044 ceph-mon[32590]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:45.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:45 smithi037 ceph-mon[29204]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:45.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:45 smithi069 ceph-mon[32534]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:45.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:45 smithi044 ceph-mon[32590]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:46.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:46 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:19:46.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:46 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:46.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:46 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:19:46.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:46 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:46.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:46 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T13:19:46.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:46 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:47 smithi037 ceph-mon[29204]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:47 smithi037 ceph-mon[29204]: Deploying daemon osd.3 on smithi037 2024-04-25T13:19:47.553 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:47 smithi069 ceph-mon[32534]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:47.553 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:47 smithi069 ceph-mon[32534]: Deploying daemon osd.3 on smithi037 2024-04-25T13:19:47.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:47 smithi044 ceph-mon[32590]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:47.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:47 smithi044 ceph-mon[32590]: Deploying daemon osd.3 on smithi037 2024-04-25T13:19:49.467 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:49 smithi037 ceph-mon[29204]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:49.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:49 smithi069 ceph-mon[32534]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:49.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:49 smithi044 ceph-mon[32590]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:51.578 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:51 smithi037 ceph-mon[29204]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:51.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:51 smithi069 ceph-mon[32534]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:51.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:51 smithi044 ceph-mon[32590]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:52 smithi037 ceph-mon[29204]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:52 smithi069 ceph-mon[32534]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:52 smithi044 ceph-mon[32590]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:55.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:55.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:55.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.611+0000 7f9fbb7fe700 1 -- 172.21.15.37:0/3337249602 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9fcc062e80 con 0x7f9fb40a3cb0 2024-04-25T13:19:55.611 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-04-25T13:19:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 msgr2=0x7f9fb40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 0x7f9fb40a6170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc005950 tx=0x7f9fbc00a300 comp rx=0 tx=0).stop 2024-04-25T13:19:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 msgr2=0x7f9fcc1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000d980 tx=0x7f9fc000dc90 comp rx=0 tx=0).stop 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 shutdown_connections 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9fb40a3cb0 0x7f9fb40a6170 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fcc1035b0 0x7f9fcc198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9fcc108810 0x7f9fcc1a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 --2- 172.21.15.37:0/3337249602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fcc105ee0 0x7f9fcc1990d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 >> 172.21.15.37:0/3337249602 conn(0x7f9fcc0fcf60 msgr2=0x7f9fcc103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 shutdown_connections 2024-04-25T13:19:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:19:55.613+0000 7f9fd0e31700 1 -- 172.21.15.37:0/3337249602 wait complete. 2024-04-25T13:19:56.257 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T13:19:56.259 INFO:tasks.cephadm:Deploying osd.4 on smithi044 with /dev/vg_nvme/lv_3... 2024-04-25T13:19:56.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:19:56.512 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:19:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:56 smithi037 ceph-mon[29204]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:56 smithi037 ceph-mon[29204]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:19:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:56 smithi069 ceph-mon[32534]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:56 smithi069 ceph-mon[32534]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:19:56.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:19:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:56 smithi044 ceph-mon[32590]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:56 smithi044 ceph-mon[32590]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T13:19:57.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:57 smithi037 ceph-mon[29204]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:19:57.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:57 smithi037 ceph-mon[29204]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:19:57.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:57 smithi037 ceph-mon[29204]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:19:57.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:57 smithi069 ceph-mon[32534]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:19:58.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:57 smithi069 ceph-mon[32534]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:19:58.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:57 smithi069 ceph-mon[32534]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:19:58.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:57 smithi044 ceph-mon[32590]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T13:19:58.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:57 smithi044 ceph-mon[32590]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T13:19:58.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:57 smithi044 ceph-mon[32590]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:19:58.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.243 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:19:58.243 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:19:58.244 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-25T13:19:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-25T13:19:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121995 s, 860 MB/s 2024-04-25T13:19:58.244 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 2024-04-25T13:19:58.244 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-25T13:19:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:19:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:19:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:58.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' 2024-04-25T13:19:58.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:19:58.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.852 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:19:58.562+0000 7f3a72557700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:19:58.853 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:19:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:19:58.689+0000 7f3a6a342700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:19:58.860 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:19:58.860 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:19:58.860 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.860 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:58.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' 2024-04-25T13:19:58.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:19:58.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.861 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:58.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T13:19:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911]' entity='osd.3' 2024-04-25T13:19:59.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T13:19:59.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:59.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:19:59.143 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi037 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:19:59.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:19:59.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:19:59.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] boot 2024-04-25T13:19:59.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:19:59.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:19:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:20:00.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:20:00.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:20:00.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi037 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] boot 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:20:00.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:19:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi037 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:00.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:00.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:00.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: osd.3 [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] boot 2024-04-25T13:20:00.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T13:20:00.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:19:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 -- 172.21.15.44:0/1636723515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f50102ba0 msgr2=0x7f9f50103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/1636723515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f50102ba0 0x7f9f50103000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9f440099c0 tx=0x7f9f44009cd0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 -- 172.21.15.44:0/1636723515 shutdown_connections 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/1636723515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f50109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/1636723515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f501019a0 0x7f9f50101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/1636723515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f50102ba0 0x7f9f50103000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 -- 172.21.15.44:0/1636723515 >> 172.21.15.44:0/1636723515 conn(0x7f9f500fcf40 msgr2=0x7f9f500ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 -- 172.21.15.44:0/1636723515 shutdown_connections 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 -- 172.21.15.44:0/1636723515 wait complete. 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.726+0000 7f9f54b3b700 1 Processor -- start 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 -- start start 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f50102ba0 0x7f9f50103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f501044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010d440 con 0x7f9f50102ba0 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010d2c0 con 0x7f9f501019a0 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f54b3b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010d5c0 con 0x7f9f50103540 2024-04-25T13:20:00.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42884/0 (socket says 172.21.15.44:42884) 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 -- 172.21.15.44:0/777669046 learned_addr learned my addr 172.21.15.44:0/777669046 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4dd9b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f50102ba0 0x7f9f50103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4ed9d700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f501044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 -- 172.21.15.44:0/777669046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 msgr2=0x7f9f501044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f501044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 -- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f50102ba0 msgr2=0x7f9f50103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f50102ba0 0x7f9f50103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 -- 172.21.15.44:0/777669046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f50104d90 con 0x7f9f501019a0 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4ed9d700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f501044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.727+0000 7f9f4e59c700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:00.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.728+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017030 con 0x7f9f501019a0 2024-04-25T13:20:00.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.728+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a2db0 con 0x7f9f501019a0 2024-04-25T13:20:00.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.728+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f400044d0 con 0x7f9f501019a0 2024-04-25T13:20:00.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.728+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400039e0 con 0x7f9f501019a0 2024-04-25T13:20:00.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.728+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a31b0 con 0x7f9f501019a0 2024-04-25T13:20:00.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.729+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f50069730 con 0x7f9f501019a0 2024-04-25T13:20:00.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.730+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9f4001a070 con 0x7f9f501019a0 2024-04-25T13:20:00.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.730+0000 7f9f3f7fe700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 0x7f9f380a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:00.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.731+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f9f400cebb0 con 0x7f9f501019a0 2024-04-25T13:20:00.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.731+0000 7f9f4dd9b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 0x7f9f380a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:00.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.731+0000 7f9f4dd9b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 0x7f9f380a6200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009990 tx=0x7f9f440036c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:00.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.736+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f4009c6f0 con 0x7f9f501019a0 2024-04-25T13:20:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:00 smithi044 ceph-mon[32590]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:00 smithi044 ceph-mon[32590]: overall HEALTH_OK 2024-04-25T13:20:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:00 smithi044 ceph-mon[32590]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:20:00.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:00.952+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9f5006ae30 con 0x7f9f380a3d40 2024-04-25T13:20:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:00 smithi037 ceph-mon[29204]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:00 smithi037 ceph-mon[29204]: overall HEALTH_OK 2024-04-25T13:20:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:00 smithi037 ceph-mon[29204]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:20:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:00 smithi069 ceph-mon[32534]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 130 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:00 smithi069 ceph-mon[32534]: overall HEALTH_OK 2024-04-25T13:20:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:00 smithi069 ceph-mon[32534]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T13:20:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:01 smithi044 ceph-mon[32590]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:01 smithi044 ceph-mon[32590]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:20:02.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:01 smithi037 ceph-mon[29204]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:02.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:02.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:02.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:02.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:01 smithi037 ceph-mon[29204]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:20:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:01 smithi069 ceph-mon[32534]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:01 smithi069 ceph-mon[32534]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T13:20:02.980 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:02 smithi044 ceph-mon[32590]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:02.980 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:02 smithi044 ceph-mon[32590]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:20:02.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:02 smithi037 ceph-mon[29204]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:02.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:02 smithi037 ceph-mon[29204]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:20:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:02 smithi069 ceph-mon[32534]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:02 smithi069 ceph-mon[32534]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T13:20:04.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:03 smithi037 ceph-mon[29204]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:20:04.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:03 smithi037 ceph-mon[29204]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:20:04.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:03 smithi069 ceph-mon[32534]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:20:04.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:03 smithi069 ceph-mon[32534]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:20:04.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:03 smithi044 ceph-mon[32590]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T13:20:04.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:03 smithi044 ceph-mon[32590]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2789835952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]': finished 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:04 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/3456121661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2789835952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]': finished 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:04 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/3456121661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2789835952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]: dispatch 2024-04-25T13:20:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "748fab07-a621-4314-a001-a402dc9d7d08"}]': finished 2024-04-25T13:20:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T13:20:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:04 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/3456121661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:06 smithi037 ceph-mon[29204]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:06 smithi069 ceph-mon[32534]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:06 smithi044 ceph-mon[32590]: pgmap v112: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T13:20:08.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:07 smithi037 ceph-mon[29204]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:20:08.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:07 smithi037 ceph-mon[29204]: Cluster is now healthy 2024-04-25T13:20:08.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:07 smithi069 ceph-mon[32534]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:20:08.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:07 smithi069 ceph-mon[32534]: Cluster is now healthy 2024-04-25T13:20:08.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:07 smithi044 ceph-mon[32590]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T13:20:08.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:07 smithi044 ceph-mon[32590]: Cluster is now healthy 2024-04-25T13:20:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:08 smithi037 ceph-mon[29204]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:20:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:08 smithi069 ceph-mon[32534]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:20:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:08 smithi044 ceph-mon[32590]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T13:20:11.065 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:10 smithi044 ceph-mon[32590]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T13:20:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:10 smithi037 ceph-mon[29204]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T13:20:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:10 smithi069 ceph-mon[32534]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-25T13:20:12.039 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:20:12.039 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:12.039 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:11 smithi044 ceph-mon[32590]: Deploying daemon osd.4 on smithi044 2024-04-25T13:20:12.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:20:12.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:12.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:11 smithi037 ceph-mon[29204]: Deploying daemon osd.4 on smithi044 2024-04-25T13:20:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T13:20:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:11 smithi069 ceph-mon[32534]: Deploying daemon osd.4 on smithi044 2024-04-25T13:20:13.044 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:12 smithi044 ceph-mon[32590]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:20:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:12 smithi037 ceph-mon[29204]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:20:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:12 smithi069 ceph-mon[32534]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:20:15.073 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:14 smithi044 ceph-mon[32590]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:20:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:14 smithi037 ceph-mon[29204]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:20:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:14 smithi069 ceph-mon[32534]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T13:20:16.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:16 smithi044 ceph-mon[32590]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:20:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:16 smithi037 ceph-mon[29204]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:20:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:16 smithi069 ceph-mon[32534]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T13:20:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:18.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:18.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:18.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:19.076 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:18 smithi044 ceph-mon[32590]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:20:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:18 smithi037 ceph-mon[29204]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:20:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:18 smithi069 ceph-mon[32534]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:20:20.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.173+0000 7f9f3f7fe700 1 -- 172.21.15.44:0/777669046 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f5006ae30 con 0x7f9f380a3d40 2024-04-25T13:20:20.174 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 4 on host 'smithi044' 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 msgr2=0x7f9f380a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 0x7f9f380a6200 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009990 tx=0x7f9f440036c0 comp rx=0 tx=0).stop 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 msgr2=0x7f9f501058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d980 tx=0x7f9f4000dc90 comp rx=0 tx=0).stop 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 shutdown_connections 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9f380a3d40 0x7f9f380a6200 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:20.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9f501019a0 0x7f9f501058e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9f50103540 0x7f9f501044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 --2- 172.21.15.44:0/777669046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f50102ba0 0x7f9f50103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 >> 172.21.15.44:0/777669046 conn(0x7f9f500fcf40 msgr2=0x7f9f5006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:20:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 shutdown_connections 2024-04-25T13:20:20.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:20:20.176+0000 7f9f54b3b700 1 -- 172.21.15.44:0/777669046 wait complete. 2024-04-25T13:20:20.858 DEBUG:teuthology.orchestra.run.smithi044:osd.4> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T13:20:20.860 INFO:tasks.cephadm:Deploying osd.5 on smithi069 with /dev/vg_nvme/lv_3... 2024-04-25T13:20:20.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T13:20:21.105 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:21.350 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:21 smithi044 ceph-mon[32590]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.408 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:21.409 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:21 smithi069 ceph-mon[32534]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:21 smithi037 ceph-mon[29204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T13:20:22.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:22 smithi044 ceph-mon[32590]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:20:22.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:22 smithi044 ceph-mon[32590]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:20:22.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:22 smithi044 ceph-mon[32590]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:22.254 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:22 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:22.255 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:22 smithi044 ceph-mon[32590]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:22.469 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:22 smithi069 ceph-mon[32534]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:20:22.470 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:22 smithi069 ceph-mon[32534]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:20:22.470 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:22 smithi069 ceph-mon[32534]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:22.470 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:22 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:22.470 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:22 smithi069 ceph-mon[32534]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:22 smithi037 ceph-mon[29204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T13:20:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:22 smithi037 ceph-mon[29204]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T13:20:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:22 smithi037 ceph-mon[29204]: from='osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:22 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:22.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:22 smithi037 ceph-mon[29204]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127291 s, 824 MB/s 2024-04-25T13:20:23.017 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T13:20:23.576 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:23.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:20:23.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:20:23.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.577 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-mon[32590]: from='osd.4 ' entity='osd.4' 2024-04-25T13:20:23.577 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4[44083]: 2024-04-25T13:20:23.160+0000 7fa10e79b700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:20:23.577 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:20:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4[44083]: 2024-04-25T13:20:23.236+0000 7fa105d85700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:23 smithi037 ceph-mon[29204]: from='osd.4 ' entity='osd.4' 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:23 smithi069 ceph-mon[32534]: from='osd.4 ' entity='osd.4' 2024-04-25T13:20:23.678 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-04-25T13:20:23.916 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:20:24.361 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:20:24.361 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:20:24.361 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:24.361 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.361 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.362 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:24 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:24 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:20:24.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:24.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:24 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:25.481 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:25.481 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi044 2024-04-25T13:20:25.481 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-25T13:20:25.481 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] boot 2024-04-25T13:20:25.481 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:20:25.482 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:25 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 -- 172.21.15.69:0/3727063630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 msgr2=0x7f78d4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 --2- 172.21.15.69:0/3727063630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4109df0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009f90 tx=0x7f78cc00b330 comp rx=0 tx=0).stop 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 -- 172.21.15.69:0/3727063630 shutdown_connections 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 --2- 172.21.15.69:0/3727063630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d41019a0 0x7f78d4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 --2- 172.21.15.69:0/3727063630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4109df0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 --2- 172.21.15.69:0/3727063630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d4102ba0 0x7f78d4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 -- 172.21.15.69:0/3727063630 >> 172.21.15.69:0/3727063630 conn(0x7f78d40fcf40 msgr2=0x7f78d40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 -- 172.21.15.69:0/3727063630 shutdown_connections 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 -- 172.21.15.69:0/3727063630 wait complete. 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.480+0000 7f78d95ca700 1 Processor -- start 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 -- start start 2024-04-25T13:20:25.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 0x7f78d406c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d4102ba0 0x7f78d406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410d440 con 0x7f78d41019a0 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410d2c0 con 0x7f78d4102ba0 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d95ca700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410d5c0 con 0x7f78d4103540 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40740/0 (socket says 172.21.15.69:40740) 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 -- 172.21.15.69:0/4196250932 learned_addr learned my addr 172.21.15.69:0/4196250932 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d27fc700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d4102ba0 0x7f78d406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d2ffd700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 0x7f78d406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 -- 172.21.15.69:0/4196250932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d4102ba0 msgr2=0x7f78d406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:25.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d4102ba0 0x7f78d406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 -- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 msgr2=0x7f78d406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 0x7f78d406c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 -- 172.21.15.69:0/4196250932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d40744f0 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d2ffd700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 0x7f78d406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.481+0000 7f78d37fe700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0079b0 tx=0x7f78cc007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.482+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc021070 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.482+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d4074780 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.482+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc01db00 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.482+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc01bc00 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.482+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41a99e0 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.483+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d410dc00 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.484+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f78cc024070 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.484+0000 7f78bbfff700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 0x7f78bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.484+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7f78cc0d0910 con 0x7f78d4103540 2024-04-25T13:20:25.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.484+0000 7f78d2ffd700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 0x7f78bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.485+0000 7f78d2ffd700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 0x7f78bc0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005950 tx=0x7f78c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:25.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.489+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78cc09e3f0 con 0x7f78d4103540 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi044 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] boot 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:20:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:25 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi044 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 8546M 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: osd.4 [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] boot 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T13:20:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:25 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:20:25.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:25.692+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f78d40fdc80 con 0x7f78bc0a3c40 2024-04-25T13:20:26.545 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:26 smithi069 ceph-mon[32534]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:20:26.545 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:26.545 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:26.545 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:26.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:26 smithi037 ceph-mon[29204]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:20:26.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:26.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:26.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:26.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:26 smithi044 ceph-mon[32590]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T13:20:26.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:26.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:26.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:27 smithi069 ceph-mon[32534]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:27 smithi069 ceph-mon[32534]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:27.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:27 smithi037 ceph-mon[29204]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:27.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:27 smithi037 ceph-mon[29204]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:27.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:27 smithi044 ceph-mon[32590]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:27.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:27 smithi044 ceph-mon[32590]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:28.576 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:28 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/3127769467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.576 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:28 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.576 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:28 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]': finished 2024-04-25T13:20:28.576 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:28 smithi069 ceph-mon[32534]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:20:28.577 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:28 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/3127769467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:28 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:28 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]': finished 2024-04-25T13:20:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:28 smithi037 ceph-mon[29204]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:20:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:28 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/3127769467' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:28 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]: dispatch 2024-04-25T13:20:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:28 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1dbcf0a2-1319-4cc8-9c58-d1d2a5520476"}]': finished 2024-04-25T13:20:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:28 smithi044 ceph-mon[32590]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T13:20:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:29.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:29 smithi037 ceph-mon[29204]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:29.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:29 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/1156935845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:29 smithi069 ceph-mon[32534]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:29.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:29 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/1156935845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:29 smithi044 ceph-mon[32590]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:29.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:29 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/1156935845' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:31.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:31 smithi037 ceph-mon[29204]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:31.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:31 smithi069 ceph-mon[32534]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:31.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:31 smithi044 ceph-mon[32590]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:33.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:33 smithi069 ceph-mon[32534]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:33 smithi037 ceph-mon[29204]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:33.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:33 smithi044 ceph-mon[32590]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:35 smithi037 ceph-mon[29204]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:35 smithi069 ceph-mon[32534]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:35.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:35 smithi044 ceph-mon[32590]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:20:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:36.605 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:20:36.605 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T13:20:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:37.591 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:37 smithi069 ceph-mon[32534]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:37.592 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:37 smithi069 ceph-mon[32534]: Deploying daemon osd.5 on smithi069 2024-04-25T13:20:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:37 smithi037 ceph-mon[29204]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:37 smithi037 ceph-mon[29204]: Deploying daemon osd.5 on smithi069 2024-04-25T13:20:37.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:37 smithi044 ceph-mon[32590]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:37.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:37 smithi044 ceph-mon[32590]: Deploying daemon osd.5 on smithi069 2024-04-25T13:20:39.544 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:39 smithi069 ceph-mon[32534]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:39 smithi037 ceph-mon[29204]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:39 smithi044 ceph-mon[32590]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:41.557 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:41 smithi069 ceph-mon[32534]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:41.557 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.557 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.557 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:41 smithi037 ceph-mon[29204]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:41 smithi044 ceph-mon[32590]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:41.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:43.541 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:43 smithi069 ceph-mon[32534]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:43 smithi037 ceph-mon[29204]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:43.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:43 smithi044 ceph-mon[32590]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:44.760 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 5 on host 'smithi069' 2024-04-25T13:20:44.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.759+0000 7f78bbfff700 1 -- 172.21.15.69:0/4196250932 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78d40fdc80 con 0x7f78bc0a3c40 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 msgr2=0x7f78bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 0x7f78bc0a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005950 tx=0x7f78c40058e0 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 msgr2=0x7f78d4073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0079b0 tx=0x7f78cc007a90 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 shutdown_connections 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78bc0a3c40 0x7f78bc0a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78d4103540 0x7f78d4073df0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d41019a0 0x7f78d406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 --2- 172.21.15.69:0/4196250932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78d4102ba0 0x7f78d406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 >> 172.21.15.69:0/4196250932 conn(0x7f78d40fcf40 msgr2=0x7f78d40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 shutdown_connections 2024-04-25T13:20:44.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:20:44.762+0000 7f78d95ca700 1 -- 172.21.15.69:0/4196250932 wait complete. 2024-04-25T13:20:45.094 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.095 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:45.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:20:45.466 DEBUG:teuthology.orchestra.run.smithi069:osd.5> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T13:20:45.469 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-25T13:20:45.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:20:45.726 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:20:46.017 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:45 smithi069 ceph-mon[32534]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:46.017 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:45 smithi069 ceph-mon[32534]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:45 smithi037 ceph-mon[29204]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:46.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:45 smithi037 ceph-mon[29204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:46.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:45 smithi044 ceph-mon[32590]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:46.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:45 smithi044 ceph-mon[32590]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T13:20:47.023 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:47.023 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:20:47.024 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:20:47.024 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:47.024 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.024 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-mon[32534]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.024 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:20:46.903+0000 7faeec4d7700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:20:47.024 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:20:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:20:46.914+0000 7faee82ca700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:46 smithi037 ceph-mon[29204]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: from='osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:46 smithi044 ceph-mon[32590]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0170721 s, 614 MB/s 2024-04-25T13:20:47.765 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T13:20:48.235 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:47 smithi069 ceph-mon[32534]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:20:48.235 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:47 smithi069 ceph-mon[32534]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:20:48.236 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.236 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:47 smithi037 ceph-mon[29204]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:20:48.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:47 smithi037 ceph-mon[29204]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:20:48.323 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.323 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:47 smithi044 ceph-mon[32590]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:20:48.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:47 smithi044 ceph-mon[32590]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T13:20:48.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:48.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-25T13:20:48.792 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:20:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:20:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:20:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] boot 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi069 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:49.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:20:49.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:20:49.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:49.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] boot 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi069 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:49.239 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: osd.5 [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] boot 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi069 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 8544M 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:49.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:20:49.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:20:50.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:49 smithi037 ceph-mon[29204]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:20:50.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:49 smithi069 ceph-mon[32534]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:20:50.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:49 smithi044 ceph-mon[32590]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T13:20:50.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 -- 172.21.15.37:0/939939342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c103540 msgr2=0x7fb24c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 --2- 172.21.15.37:0/939939342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c103540 0x7fb24c109df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb24400b0d0 tx=0x7fb24400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 -- 172.21.15.37:0/939939342 shutdown_connections 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 --2- 172.21.15.37:0/939939342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c1019a0 0x7fb24c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 --2- 172.21.15.37:0/939939342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c103540 0x7fb24c109df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 --2- 172.21.15.37:0/939939342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 0x7fb24c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 -- 172.21.15.37:0/939939342 >> 172.21.15.37:0/939939342 conn(0x7fb24c0fcf40 msgr2=0x7fb24c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 -- 172.21.15.37:0/939939342 shutdown_connections 2024-04-25T13:20:50.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.436+0000 7fb253bab700 1 -- 172.21.15.37:0/939939342 wait complete. 2024-04-25T13:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 Processor -- start 2024-04-25T13:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 -- start start 2024-04-25T13:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 0x7fb24c103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:50.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 0x7fb24c1044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d520 con 0x7fb24c1019a0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d3a0 con 0x7fb24c103540 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb253bab700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c10d6a0 con 0x7fb24c102ba0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb251947700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb251947700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59186/0 (socket says 172.21.15.37:59186) 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.437+0000 7fb251947700 1 -- 172.21.15.37:0/1240538861 learned_addr learned my addr 172.21.15.37:0/1240538861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb252148700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 0x7fb24c1044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251146700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 0x7fb24c103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 -- 172.21.15.37:0/1240538861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 msgr2=0x7fb24c103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 0x7fb24c103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 -- 172.21.15.37:0/1240538861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 msgr2=0x7fb24c1044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 0x7fb24c1044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 -- 172.21.15.37:0/1240538861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c104d70 con 0x7fb24c1019a0 2024-04-25T13:20:50.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb252148700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 0x7fb24c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb251947700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb24800b500 tx=0x7fb24800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248015020 con 0x7fb24c1019a0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb248011ce0 con 0x7fb24c1019a0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c1a2e60 con 0x7fb24c1019a0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248010d60 con 0x7fb24c1019a0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.438+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1a32e0 con 0x7fb24c1019a0 2024-04-25T13:20:50.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.439+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c069760 con 0x7fb24c1019a0 2024-04-25T13:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.440+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb248018020 con 0x7fb24c1019a0 2024-04-25T13:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.441+0000 7fb242ffd700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 0x7fb2380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.441+0000 7fb251146700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 0x7fb2380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.441+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4562+0+0 (secure 0 0 0) 0x7fb2480cf330 con 0x7fb24c1019a0 2024-04-25T13:20:50.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.441+0000 7fb251146700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 0x7fb2380a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005950 tx=0x7fb23c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:20:50.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.445+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb24c069760 con 0x7fb24c1019a0 2024-04-25T13:20:50.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:20:50.665+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb24c0ff160 con 0x7fb2380a3c40 2024-04-25T13:20:51.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:50 smithi044 ceph-mon[32590]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:20:51.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:50 smithi044 ceph-mon[32590]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:20:51.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:51.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:51.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:51.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:50 smithi037 ceph-mon[29204]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:20:51.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:50 smithi037 ceph-mon[29204]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:20:51.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:51.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:51.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:50 smithi069 ceph-mon[32534]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:20:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:50 smithi069 ceph-mon[32534]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T13:20:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:20:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:20:51.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:20:52.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:51 smithi037 ceph-mon[29204]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:52.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:51 smithi037 ceph-mon[29204]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:20:52.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:51 smithi069 ceph-mon[32534]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:52.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:51 smithi069 ceph-mon[32534]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:20:52.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:51 smithi044 ceph-mon[32590]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:20:52.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:51 smithi044 ceph-mon[32590]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T13:20:53.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:52 smithi037 ceph-mon[29204]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:53.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:52 smithi037 ceph-mon[29204]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:20:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:52 smithi069 ceph-mon[32534]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:52 smithi069 ceph-mon[32534]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:20:53.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:52 smithi044 ceph-mon[32590]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:53.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:52 smithi044 ceph-mon[32590]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T13:20:54.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:53 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]: dispatch 2024-04-25T13:20:54.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:53 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]': finished 2024-04-25T13:20:54.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:53 smithi037 ceph-mon[29204]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:20:54.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:20:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:53 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]: dispatch 2024-04-25T13:20:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:53 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]': finished 2024-04-25T13:20:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:53 smithi069 ceph-mon[32534]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:20:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:20:54.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:53 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]: dispatch 2024-04-25T13:20:54.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:53 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/335664593' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7fef8d2-092c-445a-aa3e-64878cfcd1a3"}]': finished 2024-04-25T13:20:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:53 smithi044 ceph-mon[32590]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T13:20:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:20:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:54 smithi037 ceph-mon[29204]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/728236110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:55.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:54 smithi069 ceph-mon[32534]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:55.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:54 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/728236110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:55.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:54 smithi044 ceph-mon[32590]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T13:20:55.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:54 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/728236110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:20:57.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:56 smithi037 ceph-mon[29204]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T13:20:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:56 smithi069 ceph-mon[32534]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T13:20:57.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:56 smithi044 ceph-mon[32590]: pgmap v150: 1 pgs: 1 peering; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T13:20:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:20:58 smithi037 ceph-mon[29204]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T13:20:59.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:20:58 smithi069 ceph-mon[32534]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T13:20:59.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:20:58 smithi044 ceph-mon[32590]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T13:21:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:00 smithi037 ceph-mon[29204]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:00 smithi069 ceph-mon[32534]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:01.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:00 smithi044 ceph-mon[32590]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:02.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:21:02.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:02.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:01 smithi037 ceph-mon[29204]: Deploying daemon osd.6 on smithi037 2024-04-25T13:21:02.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:21:02.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:02.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:01 smithi069 ceph-mon[32534]: Deploying daemon osd.6 on smithi037 2024-04-25T13:21:02.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T13:21:02.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:02.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:01 smithi044 ceph-mon[32590]: Deploying daemon osd.6 on smithi037 2024-04-25T13:21:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:02 smithi037 ceph-mon[29204]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:02 smithi069 ceph-mon[32534]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:03.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:02 smithi044 ceph-mon[32590]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:05.206 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:04 smithi037 ceph-mon[29204]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:04 smithi069 ceph-mon[32534]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:04 smithi044 ceph-mon[32590]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:07.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:06 smithi037 ceph-mon[29204]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:07.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:07.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:06 smithi069 ceph-mon[32534]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:07.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:06 smithi044 ceph-mon[32590]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:08 smithi037 ceph-mon[29204]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:09.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:08 smithi069 ceph-mon[32534]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:09.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:08 smithi044 ceph-mon[32590]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:10.026 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-04-25T13:21:10.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.025+0000 7fb242ffd700 1 -- 172.21.15.37:0/1240538861 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb24c0ff160 con 0x7fb2380a3c40 2024-04-25T13:21:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 msgr2=0x7fb2380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 0x7fb2380a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb23c005950 tx=0x7fb23c005cf0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 msgr2=0x7fb24c105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb24800b500 tx=0x7fb24800b8c0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 shutdown_connections 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2380a3c40 0x7fb2380a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb24c103540 0x7fb24c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb24c1019a0 0x7fb24c105930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 --2- 172.21.15.37:0/1240538861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb24c102ba0 0x7fb24c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.027+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 >> 172.21.15.37:0/1240538861 conn(0x7fb24c0fcf40 msgr2=0x7fb24c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.028+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 shutdown_connections 2024-04-25T13:21:10.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:21:10.028+0000 7fb253bab700 1 -- 172.21.15.37:0/1240538861 wait complete. 2024-04-25T13:21:11.158 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T13:21:11.160 INFO:tasks.cephadm:Deploying osd.7 on smithi044 with /dev/vg_nvme/lv_2... 2024-04-25T13:21:11.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:21:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:10 smithi037 ceph-mon[29204]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:10 smithi037 ceph-mon[29204]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:21:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:10 smithi069 ceph-mon[32534]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:10 smithi069 ceph-mon[32534]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:21:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:10 smithi044 ceph-mon[32590]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:10 smithi044 ceph-mon[32590]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T13:21:11.418 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:12.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:12.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:21:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:21:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:13 smithi044 ceph-mon[32590]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-mon[29204]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' 2024-04-25T13:21:13.267 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:21:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:21:12.946+0000 7f7afa24a700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:21:13.267 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:21:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:21:13.009+0000 7f7af2035700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:13 smithi069 ceph-mon[32534]: from='osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223]' entity='osd.6' 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120429 s, 871 MB/s 2024-04-25T13:21:13.445 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-25T13:21:14.039 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:14 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:21:14.040 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:14 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:21:14.040 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:14 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:14.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-04-25T13:21:14.331 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:21:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:14 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:21:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:14 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:21:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:14 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:14.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:14 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:21:14.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:14 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:21:14.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:14 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] boot 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:15.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:15 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] boot 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:15.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:15 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.388 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-04-25T13:21:15.388 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.388 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: osd.6 [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] boot 2024-04-25T13:21:15.388 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:15 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 -- 172.21.15.44:0/3298403836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638103540 msgr2=0x7f7638109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 --2- 172.21.15.44:0/3298403836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638103540 0x7f7638109df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f762800b0d0 tx=0x7f762800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 -- 172.21.15.44:0/3298403836 shutdown_connections 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 --2- 172.21.15.44:0/3298403836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638103540 0x7f7638109df0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 --2- 172.21.15.44:0/3298403836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76381019a0 0x7f7638101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 --2- 172.21.15.44:0/3298403836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7638102ba0 0x7f7638103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 -- 172.21.15.44:0/3298403836 >> 172.21.15.44:0/3298403836 conn(0x7f76380fcf40 msgr2=0x7f76380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 -- 172.21.15.44:0/3298403836 shutdown_connections 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 -- 172.21.15.44:0/3298403836 wait complete. 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.859+0000 7f763df9c700 1 Processor -- start 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 -- start start 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76381019a0 0x7f7638198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7638103540 0x7f76381a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f763810d440 con 0x7f76381019a0 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f763810d2c0 con 0x7f7638102ba0 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f763df9c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f763810d5c0 con 0x7f7638103540 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58640/0 (socket says 172.21.15.44:58640) 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 -- 172.21.15.44:0/4119850728 learned_addr learned my addr 172.21.15.44:0/4119850728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-25T13:21:15.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f76377fe700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76381019a0 0x7f7638198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7637fff700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7638103540 0x7f76381a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 -- 172.21.15.44:0/4119850728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7638103540 msgr2=0x7f76381a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7638103540 0x7f76381a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 -- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76381019a0 msgr2=0x7f7638198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76381019a0 0x7f7638198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.860+0000 7f7636ffd700 1 -- 172.21.15.44:0/4119850728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76381a0820 con 0x7f7638102ba0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f7636ffd700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f762000c920 tx=0x7f762000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620004a30 con 0x7f7638102ba0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76200054d0 con 0x7f7638102ba0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620007cc0 con 0x7f7638102ba0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76381a0b10 con 0x7f7638102ba0 2024-04-25T13:21:15.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.861+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76381a1050 con 0x7f7638102ba0 2024-04-25T13:21:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.862+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7638069730 con 0x7f7638102ba0 2024-04-25T13:21:15.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.863+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7620004f30 con 0x7f7638102ba0 2024-04-25T13:21:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.864+0000 7f7634ff9700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 0x7f76240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.864+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f76200cf4f0 con 0x7f7638102ba0 2024-04-25T13:21:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.864+0000 7f76377fe700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 0x7f76240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:15.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.864+0000 7f76377fe700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 0x7f76240a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7638102a00 tx=0x7f763000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:21:15.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:15.867+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7620014030 con 0x7f7638102ba0 2024-04-25T13:21:16.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:16.088+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f76380ff140 con 0x7f76240a3c40 2024-04-25T13:21:16.286 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi037 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:16.287 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:16 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi037 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:16.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:16 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi037 2024-04-25T13:21:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T13:21:16.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T13:21:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:16.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:16 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:17 smithi037 ceph-mon[29204]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:17 smithi037 ceph-mon[29204]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:17 smithi069 ceph-mon[32534]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:17 smithi069 ceph-mon[32534]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:17.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:17 smithi044 ceph-mon[32590]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:17.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:17 smithi044 ceph-mon[32590]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:17.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:17.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:17.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2879303996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]': finished 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:21:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2879303996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]': finished 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:21:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2879303996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]: dispatch 2024-04-25T13:21:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb"}]': finished 2024-04-25T13:21:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T13:21:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:20.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:20 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2851373145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:20.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:20 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2851373145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:20.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:20 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2851373145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:21.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:21 smithi044 ceph-mon[32590]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:21.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:21 smithi037 ceph-mon[29204]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:21.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:21 smithi069 ceph-mon[32534]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:23 smithi037 ceph-mon[29204]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:23.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:23 smithi069 ceph-mon[32534]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:23.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:23 smithi044 ceph-mon[32590]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:25.275 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:25 smithi044 ceph-mon[32590]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:25.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:25 smithi037 ceph-mon[29204]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:25.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:25 smithi069 ceph-mon[32534]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:27.282 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:27 smithi044 ceph-mon[32590]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:27.282 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:21:27.282 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:27.283 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:27 smithi044 ceph-mon[32590]: Deploying daemon osd.7 on smithi044 2024-04-25T13:21:27.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:27 smithi037 ceph-mon[29204]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:27.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:21:27.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:27.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:27 smithi037 ceph-mon[29204]: Deploying daemon osd.7 on smithi044 2024-04-25T13:21:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:27 smithi069 ceph-mon[32534]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T13:21:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:27.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:27 smithi069 ceph-mon[32534]: Deploying daemon osd.7 on smithi044 2024-04-25T13:21:29.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:29 smithi037 ceph-mon[29204]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:29.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:29 smithi069 ceph-mon[32534]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:29.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:29 smithi044 ceph-mon[32590]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:31.292 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:31 smithi044 ceph-mon[32590]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:31.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:31 smithi037 ceph-mon[29204]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:31.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:31 smithi069 ceph-mon[32534]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:32 smithi037 ceph-mon[29204]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:32 smithi069 ceph-mon[32534]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:33.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:32 smithi044 ceph-mon[32590]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:33.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:35.341 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:34 smithi044 ceph-mon[32590]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:35.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:34 smithi037 ceph-mon[29204]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:35.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:34 smithi069 ceph-mon[32534]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:35.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.405+0000 7f7634ff9700 1 -- 172.21.15.44:0/4119850728 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76380ff140 con 0x7f76240a3c40 2024-04-25T13:21:35.405 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 7 on host 'smithi044' 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 msgr2=0x7f76240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 0x7f76240a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7638102a00 tx=0x7f763000b410 comp rx=0 tx=0).stop 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 msgr2=0x7f7638199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f762000c920 tx=0x7f762000cc30 comp rx=0 tx=0).stop 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 shutdown_connections 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f76240a3c40 0x7f76240a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:35.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7638103540 0x7f76381a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:35.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76381019a0 0x7f7638198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:35.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 --2- 172.21.15.44:0/4119850728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7638102ba0 0x7f7638199100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:35.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 >> 172.21.15.44:0/4119850728 conn(0x7f76380fcf40 msgr2=0x7f76380fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:21:35.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 shutdown_connections 2024-04-25T13:21:35.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:21:35.407+0000 7f763df9c700 1 -- 172.21.15.44:0/4119850728 wait complete. 2024-04-25T13:21:35.994 DEBUG:teuthology.orchestra.run.smithi044:osd.7> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T13:21:35.997 INFO:tasks.cephadm:Deploying osd.8 on smithi069 with /dev/vg_nvme/lv_2... 2024-04-25T13:21:35.997 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T13:21:36.245 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T13:21:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:36.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:37.517 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:37 smithi069 ceph-mon[32534]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:37.651 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:37.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:21:37.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:37.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:21:37.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:37.652 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:37 smithi044 ceph-mon[32590]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T13:21:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: from='osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T13:21:37.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:37.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:37 smithi037 ceph-mon[29204]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159305 s, 658 MB/s 2024-04-25T13:21:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T13:21:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:21:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:21:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='osd.7 ' entity='osd.7' 2024-04-25T13:21:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T13:21:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:21:38.330+0000 7fea5cca7700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:21:38.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:21:38.461+0000 7fea54a92700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:21:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:21:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:21:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='osd.7 ' entity='osd.7' 2024-04-25T13:21:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T13:21:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.641 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.749 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='osd.7 ' entity='osd.7' 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:38.993 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:21:39.818 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:21:39.819 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:21:39.819 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:39.819 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] boot 2024-04-25T13:21:39.819 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:21:39.819 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] boot 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:21:39.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:39.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:21:39.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:21:39.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:39.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: osd.7 [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] boot 2024-04-25T13:21:39.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T13:21:39.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:21:40.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- 172.21.15.69:0/1911893199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0102b90 msgr2=0x7fb2f0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 --2- 172.21.15.69:0/1911893199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0102b90 0x7fb2f0102ff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009990 tx=0x7fb2e4009ca0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- 172.21.15.69:0/1911893199 shutdown_connections 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 --2- 172.21.15.69:0/1911893199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0101990 0x7fb2f0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 --2- 172.21.15.69:0/1911893199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0103530 0x7fb2f0109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 --2- 172.21.15.69:0/1911893199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0102b90 0x7fb2f0102ff0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- 172.21.15.69:0/1911893199 >> 172.21.15.69:0/1911893199 conn(0x7fb2f00fcf30 msgr2=0x7fb2f00ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- 172.21.15.69:0/1911893199 shutdown_connections 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- 172.21.15.69:0/1911893199 wait complete. 2024-04-25T13:21:40.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 Processor -- start 2024-04-25T13:21:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.615+0000 7fb2f5666700 1 -- start start 2024-04-25T13:21:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0102b90 0x7fb2f0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0103530 0x7fb2f01044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:40.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d400 con 0x7fb2f0102b90 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d280 con 0x7fb2f0103530 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d580 con 0x7fb2f0101990 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40078/0 (socket says 172.21.15.69:40078) 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 -- 172.21.15.69:0/2789168828 learned_addr learned my addr 172.21.15.69:0/2789168828 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2ef7fe700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0103530 0x7fb2f01044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2ee7fc700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0102b90 0x7fb2f0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 -- 172.21.15.69:0/2789168828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0103530 msgr2=0x7fb2f01044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0103530 0x7fb2f01044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 -- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0102b90 msgr2=0x7fb2f0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0102b90 0x7fb2f0103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 -- 172.21.15.69:0/2789168828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0104d80 con 0x7fb2f0101990 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2eeffd700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e970 tx=0x7fb2e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:21:40.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000cbd0 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e00044d0 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a2d00 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003df0 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.616+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a31b0 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.617+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f01a33e0 con 0x7fb2f0101990 2024-04-25T13:21:40.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.619+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb2e0003f90 con 0x7fb2f0101990 2024-04-25T13:21:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.620+0000 7fb2d7fff700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 0x7fb2d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:21:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.620+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7fb2e00cfba0 con 0x7fb2f0101990 2024-04-25T13:21:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.620+0000 7fb2ee7fc700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 0x7fb2d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:21:40.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.621+0000 7fb2ee7fc700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 0x7fb2d80a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400b2e0 tx=0x7fb2e4009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:21:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.623+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2e009d150 con 0x7fb2f0101990 2024-04-25T13:21:40.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:21:40.829+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb2f01a3670 con 0x7fb2d80a3cc0 2024-04-25T13:21:40.830 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi044 2024-04-25T13:21:40.830 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.831 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:40 smithi069 ceph-mon[32534]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi044 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:40 smithi037 ceph-mon[29204]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi044 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 5697M 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:40.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:40.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:40.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:40.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:40 smithi044 ceph-mon[32590]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T13:21:41.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:41 smithi069 ceph-mon[32534]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:41.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:41.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:41.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:41.800 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:41 smithi069 ceph-mon[32534]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:21:41.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:41 smithi037 ceph-mon[29204]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:41.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:41.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:41.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:41.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:41 smithi037 ceph-mon[29204]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:21:41.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:41 smithi044 ceph-mon[32590]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T13:21:41.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:21:41.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:21:41.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:41.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:41 smithi044 ceph-mon[32590]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T13:21:42.732 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:42 smithi069 ceph-mon[32534]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:42.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:42 smithi037 ceph-mon[29204]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:42.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:42 smithi044 ceph-mon[32590]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:21:43.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:43 smithi037 ceph-mon[29204]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:43.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:43 smithi037 ceph-mon[29204]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:21:43.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:43 smithi069 ceph-mon[32534]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:43.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:43 smithi069 ceph-mon[32534]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:21:43.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:43 smithi044 ceph-mon[32590]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:43.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:43 smithi044 ceph-mon[32590]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T13:21:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T13:21:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/2076526327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]': finished 2024-04-25T13:21:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:21:44.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:21:44.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:44 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/3474091193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/2076526327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]': finished 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:21:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:44 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/3474091193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:44.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T13:21:44.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/2076526327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]: dispatch 2024-04-25T13:21:44.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5"}]': finished 2024-04-25T13:21:44.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T13:21:44.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:21:44.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:44 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/3474091193' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:21:45.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:45 smithi037 ceph-mon[29204]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:45.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:45 smithi069 ceph-mon[32534]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:45.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:45 smithi044 ceph-mon[32590]: pgmap v188: 1 pgs: 1 remapped+peering; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:46.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:46 smithi037 ceph-mon[29204]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:46.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:46 smithi069 ceph-mon[32534]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:46.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:46 smithi044 ceph-mon[32590]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:48.928 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:48 smithi069 ceph-mon[32534]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:48 smithi037 ceph-mon[29204]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:48 smithi044 ceph-mon[32590]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:50 smithi037 ceph-mon[29204]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:50 smithi069 ceph-mon[32534]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:50 smithi044 ceph-mon[32590]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:51.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:51 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:21:51.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:51 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:51.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:51 smithi069 ceph-mon[32534]: Deploying daemon osd.8 on smithi069 2024-04-25T13:21:52.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:51 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:21:52.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:51 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:52.041 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:51 smithi044 ceph-mon[32590]: Deploying daemon osd.8 on smithi069 2024-04-25T13:21:52.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:51 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T13:21:52.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:51 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:52.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:51 smithi037 ceph-mon[29204]: Deploying daemon osd.8 on smithi069 2024-04-25T13:21:52.882 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:52 smithi069 ceph-mon[32534]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:53.057 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:52 smithi044 ceph-mon[32590]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:52 smithi037 ceph-mon[29204]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:54.850 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:54 smithi069 ceph-mon[32534]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:54 smithi037 ceph-mon[29204]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:54 smithi044 ceph-mon[32590]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:57.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:57 smithi069 ceph-mon[32534]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:57.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:57.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:57 smithi044 ceph-mon[32590]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:57.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:57 smithi037 ceph-mon[29204]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:59.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:21:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:59.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:59.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:59.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:21:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:21:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:21:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:00.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.149+0000 7fb2d7fff700 1 -- 172.21.15.69:0/2789168828 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb2f01a3670 con 0x7fb2d80a3cc0 2024-04-25T13:22:00.150 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 8 on host 'smithi069' 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 msgr2=0x7fb2d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 0x7fb2d80a6180 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400b2e0 tx=0x7fb2e4009510 comp rx=0 tx=0).stop 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 msgr2=0x7fb2f01058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000e970 tx=0x7fb2e000ec80 comp rx=0 tx=0).stop 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 shutdown_connections 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb2d80a3cc0 0x7fb2d80a6180 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f0103530 0x7fb2f01044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:00.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f0101990 0x7fb2f01058d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 --2- 172.21.15.69:0/2789168828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f0102b90 0x7fb2f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.151+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 >> 172.21.15.69:0/2789168828 conn(0x7fb2f00fcf30 msgr2=0x7fb2f00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.152+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 shutdown_connections 2024-04-25T13:22:00.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:00.152+0000 7fb2f5666700 1 -- 172.21.15.69:0/2789168828 wait complete. 2024-04-25T13:22:00.779 DEBUG:teuthology.orchestra.run.smithi069:osd.8> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T13:22:00.781 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-25T13:22:00.781 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:22:01.039 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:22:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:01.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:01 smithi069 ceph-mon[32534]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:01.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:01 smithi037 ceph-mon[29204]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:01.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:01.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:01.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:01.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:01.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:01 smithi044 ceph-mon[32590]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T13:22:02.120 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:02 smithi069 ceph-mon[32534]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:22:02.120 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:02 smithi069 ceph-mon[32534]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:22:02.120 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:02 smithi069 ceph-mon[32534]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:02.120 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:02 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:02.120 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:02 smithi069 ceph-mon[32534]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:02.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:02 smithi044 ceph-mon[32590]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:22:02.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:02 smithi044 ceph-mon[32590]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:22:02.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:02 smithi044 ceph-mon[32590]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:02.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:02 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:02.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:02 smithi044 ceph-mon[32590]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:02.478 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:02 smithi037 ceph-mon[29204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T13:22:02.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:02 smithi037 ceph-mon[29204]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T13:22:02.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:02 smithi037 ceph-mon[29204]: from='osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:02.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:02 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:02.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:02 smithi037 ceph-mon[29204]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126521 s, 829 MB/s 2024-04-25T13:22:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T13:22:03.378 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:03.378 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:22:03.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T13:22:03.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.379 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:03 smithi037 ceph-mon[29204]: from='osd.8 ' entity='osd.8' 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-mon[32534]: from='osd.8 ' entity='osd.8' 2024-04-25T13:22:03.386 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:22:03.035+0000 7f3607cbf700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T13:22:03.386 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:22:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:22:03.177+0000 7f35ffaaa700 -1 osd.8 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:22:03.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:03.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:22:03.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T13:22:03.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:03.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:03 smithi044 ceph-mon[32590]: from='osd.8 ' entity='osd.8' 2024-04-25T13:22:03.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-25T13:22:04.008 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:22:04.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:22:04.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:22:04.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: osdmap e62: 9 total, 8 up, 9 in 2024-04-25T13:22:04.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:22:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:22:04.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: osdmap e62: 9 total, 8 up, 9 in 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: osdmap e62: 9 total, 8 up, 9 in 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:04.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:04.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:05.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:05.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi069 2024-04-25T13:22:05.316 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T13:22:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] boot 2024-04-25T13:22:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:22:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:05.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:05 smithi037 ceph-mon[29204]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi069 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] boot 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:05 smithi069 ceph-mon[32534]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:22:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-04-25T13:22:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi069 2024-04-25T13:22:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 5696M 2024-04-25T13:22:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: osd.8 [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] boot 2024-04-25T13:22:05.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T13:22:05.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:22:05.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:05 smithi044 ceph-mon[32590]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 -- 172.21.15.37:0/2547584812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0104660 msgr2=0x7f95e010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/2547584812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0104660 0x7f95e010af10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95d800b0d0 tx=0x7f95d800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 -- 172.21.15.37:0/2547584812 shutdown_connections 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/2547584812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0104660 0x7f95e010af10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/2547584812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0102ac0 0x7f95e0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/2547584812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0103cc0 0x7f95e0104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 -- 172.21.15.37:0/2547584812 >> 172.21.15.37:0/2547584812 conn(0x7f95e00fe060 msgr2=0x7f95e01004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 -- 172.21.15.37:0/2547584812 shutdown_connections 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 -- 172.21.15.37:0/2547584812 wait complete. 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.506+0000 7f95e5fe4700 1 Processor -- start 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 -- start start 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0102ac0 0x7f95e0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0104660 0x7f95e01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95e010e540 con 0x7f95e0103cc0 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95e010e3c0 con 0x7f95e0102ac0 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95e5fe4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f95e010e6c0 con 0x7f95e0104660 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42456/0 (socket says 172.21.15.37:42456) 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 -- 172.21.15.37:0/3533878858 learned_addr learned my addr 172.21.15.37:0/3533878858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95dffff700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0104660 0x7f95e01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:05.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95df7fe700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0102ac0 0x7f95e0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 -- 172.21.15.37:0/3533878858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0104660 msgr2=0x7f95e01a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0104660 0x7f95e01a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 -- 172.21.15.37:0/3533878858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0102ac0 msgr2=0x7f95e0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0102ac0 0x7f95e0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 -- 172.21.15.37:0/3533878858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e01a0790 con 0x7f95e0103cc0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.507+0000 7f95deffd700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95d400d980 tx=0x7f95d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.508+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4016070 con 0x7f95e0103cc0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.508+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e01a0a80 con 0x7f95e0103cc0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.508+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d400de70 con 0x7f95e0103cc0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.508+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400f3c0 con 0x7f95e0103cc0 2024-04-25T13:22:05.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.508+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e01a1010 con 0x7f95e0103cc0 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.509+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e0069730 con 0x7f95e0103cc0 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.510+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f95d4019070 con 0x7f95e0103cc0 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.510+0000 7f95dcff9700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 0x7f95c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.511+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5858+0+0 (secure 0 0 0) 0x7f95d40cf630 con 0x7f95e0103cc0 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.511+0000 7f95df7fe700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 0x7f95c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:05.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.511+0000 7f95df7fe700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 0x7f95c80a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95e0103b20 tx=0x7f95d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:05.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.514+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95d409c900 con 0x7f95e0103cc0 2024-04-25T13:22:05.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:05.733+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f95e0062f40 con 0x7f95c80a3c40 2024-04-25T13:22:06.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:06.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:06.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:06.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:06 smithi037 ceph-mon[29204]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:22:06.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:06.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:06.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:06.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:06 smithi069 ceph-mon[32534]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:22:06.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:06.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:06 smithi044 ceph-mon[32590]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T13:22:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:07 smithi037 ceph-mon[29204]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:07 smithi037 ceph-mon[29204]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:07 smithi069 ceph-mon[32534]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:07 smithi069 ceph-mon[32534]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:07.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:07 smithi044 ceph-mon[32590]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:07.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:07 smithi044 ceph-mon[32590]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:08.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:08 smithi037 ceph-mon[29204]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T13:22:08.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:08 smithi069 ceph-mon[32534]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T13:22:08.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:08 smithi044 ceph-mon[32590]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: osdmap e67: 9 total, 9 up, 9 in 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]: dispatch 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]': finished 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:22:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: osdmap e67: 9 total, 9 up, 9 in 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]: dispatch 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]': finished 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:22:09.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:09.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:09.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: osdmap e67: 9 total, 9 up, 9 in 2024-04-25T13:22:09.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]: dispatch 2024-04-25T13:22:09.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/920507328' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c7a5005-401a-4bcb-b6fc-357793f24756"}]': finished 2024-04-25T13:22:09.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T13:22:09.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:10.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:10 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2723608287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:10.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:10 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2723608287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:10.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:10 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2723608287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:11.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:11 smithi037 ceph-mon[29204]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:11.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:11 smithi069 ceph-mon[32534]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:11.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:11 smithi044 ceph-mon[32590]: pgmap v210: 1 pgs: 1 remapped+peering; 577 KiB data, 290 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:13.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:13 smithi037 ceph-mon[29204]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:22:13.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:13 smithi069 ceph-mon[32534]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:22:13.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:13 smithi044 ceph-mon[32590]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:22:15.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:15 smithi037 ceph-mon[29204]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:22:15.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:15 smithi069 ceph-mon[32534]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:22:15.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:15 smithi044 ceph-mon[32590]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-25T13:22:17.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:17 smithi037 ceph-mon[29204]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:22:17.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:22:17.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:17.437 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:17 smithi069 ceph-mon[32534]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:22:17.437 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:22:17.437 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:17 smithi044 ceph-mon[32590]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:22:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T13:22:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:18.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:18 smithi037 ceph-mon[29204]: Deploying daemon osd.9 on smithi037 2024-04-25T13:22:18.449 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:18 smithi069 ceph-mon[32534]: Deploying daemon osd.9 on smithi037 2024-04-25T13:22:18.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:18 smithi044 ceph-mon[32590]: Deploying daemon osd.9 on smithi037 2024-04-25T13:22:19.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:19 smithi037 ceph-mon[29204]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:22:19.618 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:19 smithi069 ceph-mon[32534]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:22:19.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:19 smithi044 ceph-mon[32590]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:22:21.342 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:21 smithi037 ceph-mon[29204]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T13:22:21.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:21 smithi069 ceph-mon[32534]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T13:22:21.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:21 smithi044 ceph-mon[32590]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-04-25T13:22:23.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:23 smithi037 ceph-mon[29204]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:22:23.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:23 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:23 smithi069 ceph-mon[32534]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:22:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:23 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:23 smithi044 ceph-mon[32590]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:22:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:23.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:23 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:25 smithi037 ceph-mon[29204]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:25.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:25 smithi069 ceph-mon[32534]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:25.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:25 smithi044 ceph-mon[32590]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:25.778 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-04-25T13:22:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.778+0000 7f95dcff9700 1 -- 172.21.15.37:0/3533878858 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f95e0062f40 con 0x7f95c80a3c40 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 msgr2=0x7f95c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 0x7f95c80a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95e0103b20 tx=0x7f95d0009260 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 msgr2=0x7f95e0199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95d400d980 tx=0x7f95d400dc90 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 shutdown_connections 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95c80a3c40 0x7f95c80a6100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95e0102ac0 0x7f95e0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95e0104660 0x7f95e01a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 --2- 172.21.15.37:0/3533878858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95e0103cc0 0x7f95e0199070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.780+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 >> 172.21.15.37:0/3533878858 conn(0x7f95e00fe060 msgr2=0x7f95e0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.781+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 shutdown_connections 2024-04-25T13:22:25.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:22:25.781+0000 7f95e5fe4700 1 -- 172.21.15.37:0/3533878858 wait complete. 2024-04-25T13:22:26.372 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T13:22:26.374 INFO:tasks.cephadm:Deploying osd.10 on smithi044 with /dev/vg_nvme/lv_1... 2024-04-25T13:22:26.375 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:22:26.642 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:22:27.003 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:27.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:22:27.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.004 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:26 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:26 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T13:22:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:27.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:26 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:28.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:27 smithi044 ceph-mon[32590]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:22:28.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:27 smithi044 ceph-mon[32590]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T13:22:28.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:27 smithi044 ceph-mon[32590]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:22:28.034 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:27 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:28.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:27 smithi037 ceph-mon[29204]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:22:28.052 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:27 smithi037 ceph-mon[29204]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T13:22:28.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:27 smithi037 ceph-mon[29204]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:22:28.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:27 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:27 smithi069 ceph-mon[32534]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T13:22:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:27 smithi069 ceph-mon[32534]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T13:22:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:27 smithi069 ceph-mon[32534]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T13:22:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:27 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123435 s, 849 MB/s 2024-04-25T13:22:28.725 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: osdmap e70: 10 total, 9 up, 10 in 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' 2024-04-25T13:22:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.102 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:22:28.649+0000 7f72e3b40700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T13:22:29.102 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:22:28 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:22:28.752+0000 7f72db92b700 -1 osd.9 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: osdmap e70: 10 total, 9 up, 10 in 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' 2024-04-25T13:22:29.106 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: osdmap e70: 10 total, 9 up, 10 in 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: from='osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085]' entity='osd.9' 2024-04-25T13:22:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:29.699 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-04-25T13:22:29.958 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:22:29.997 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:29 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:22:29.997 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:29 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:22:29.997 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:29 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:29 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:22:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:29 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:22:30.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:29 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:30.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:29 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:22:30.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:29 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:22:30.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:29 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:31.022 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:31.022 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] boot 2024-04-25T13:22:31.022 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi037 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:31.023 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.024 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.024 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:30 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] boot 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi037 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:30 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 804 GiB / 805 GiB avail 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: osd.9 [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] boot 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi037 2024-04-25T13:22:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:31.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:30 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 -- 172.21.15.44:0/3250718986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4102aa0 msgr2=0x7fd3f4102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/3250718986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4102ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40099c0 tx=0x7fd3e4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 -- 172.21.15.44:0/3250718986 shutdown_connections 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/3250718986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4102ec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/3250718986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4104640 0x7fd3f410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/3250718986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 0x7fd3f4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 -- 172.21.15.44:0/3250718986 >> 172.21.15.44:0/3250718986 conn(0x7fd3f40fe040 msgr2=0x7fd3f4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 -- 172.21.15.44:0/3250718986 shutdown_connections 2024-04-25T13:22:31.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.649+0000 7fd3f8a64700 1 -- 172.21.15.44:0/3250718986 wait complete. 2024-04-25T13:22:31.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 Processor -- start 2024-04-25T13:22:31.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 -- start start 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 0x7fd3f41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410e510 con 0x7fd3f4103ca0 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410e390 con 0x7fd3f4104640 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.650+0000 7fd3f8a64700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410e690 con 0x7fd3f4102aa0 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53714/0 (socket says 172.21.15.44:53714) 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 -- 172.21.15.44:0/1058289507 learned_addr learned my addr 172.21.15.44:0/1058289507 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f1d9b700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 0x7fd3f41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f259c700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:31.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 -- 172.21.15.44:0/1058289507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4102aa0 msgr2=0x7fd3f4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 -- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 msgr2=0x7fd3f41991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 0x7fd3f41991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 -- 172.21.15.44:0/1058289507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41a0860 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f2d9d700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec007ee0 tx=0x7fd3ec00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00dbd0 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec00dd70 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41a0b50 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec005990 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.651+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a1040 con 0x7fd3f4104640 2024-04-25T13:22:31.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.652+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f4069730 con 0x7fd3f4104640 2024-04-25T13:22:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.653+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd3ec01a020 con 0x7fd3f4104640 2024-04-25T13:22:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.654+0000 7fd3e37fe700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.654+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6261+0+0 (secure 0 0 0) 0x7fd3ec0cfa70 con 0x7fd3f4104640 2024-04-25T13:22:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.654+0000 7fd3f259c700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:31.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.655+0000 7fd3f259c700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:31.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.658+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3ec010070 con 0x7fd3f4104640 2024-04-25T13:22:31.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:31.877+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd3f4062e80 con 0x7fd3dc0a3c40 2024-04-25T13:22:32.520 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:32 smithi044 ceph-mon[32590]: osdmap e72: 10 total, 10 up, 10 in 2024-04-25T13:22:32.520 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:32.520 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:32.520 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:32 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:32.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:32 smithi037 ceph-mon[29204]: osdmap e72: 10 total, 10 up, 10 in 2024-04-25T13:22:32.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:32.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:32.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:32 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:32.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:32 smithi069 ceph-mon[32534]: osdmap e72: 10 total, 10 up, 10 in 2024-04-25T13:22:32.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:32.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:32.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:32 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:33.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:33 smithi044 ceph-mon[32590]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:33 smithi044 ceph-mon[32590]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:33 smithi037 ceph-mon[29204]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:33 smithi037 ceph-mon[29204]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:33 smithi069 ceph-mon[32534]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:33 smithi069 ceph-mon[32534]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2871593106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]': finished 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:22:35.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:35 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2871593106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]': finished 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:22:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:35 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2871593106' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]: dispatch 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af8e1072-54de-4e47-9ca4-7b9fdcae474a"}]': finished 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T13:22:35.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:35 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:36 smithi037 ceph-mon[29204]: from='client.? 172.21.15.44:0/2067611498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:36 smithi069 ceph-mon[32534]: from='client.? 172.21.15.44:0/2067611498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:36.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:36 smithi044 ceph-mon[32590]: from='client.? 172.21.15.44:0/2067611498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:22:37.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:37 smithi037 ceph-mon[29204]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:37.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:37 smithi069 ceph-mon[32534]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:37.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:37 smithi044 ceph-mon[32590]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:39.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:39 smithi037 ceph-mon[29204]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:39.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:39 smithi069 ceph-mon[32534]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:39.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:39 smithi044 ceph-mon[32590]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:41.578 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:41 smithi044 ceph-mon[32590]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:41 smithi037 ceph-mon[29204]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:41.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:41 smithi069 ceph-mon[32534]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:43.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:43 smithi044 ceph-mon[32590]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:43.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:43 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:22:43.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:43 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:43.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:43 smithi044 ceph-mon[32590]: Deploying daemon osd.10 on smithi044 2024-04-25T13:22:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:43 smithi037 ceph-mon[29204]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:43 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:22:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:43 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:43 smithi037 ceph-mon[29204]: Deploying daemon osd.10 on smithi044 2024-04-25T13:22:43.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:43 smithi069 ceph-mon[32534]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:43.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:43 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T13:22:43.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:43 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:43.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:43 smithi069 ceph-mon[32534]: Deploying daemon osd.10 on smithi044 2024-04-25T13:22:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:45 smithi044 ceph-mon[32590]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:45.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:45 smithi037 ceph-mon[29204]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:45.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:45 smithi069 ceph-mon[32534]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:47.466 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:47 smithi044 ceph-mon[32590]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:47.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:47 smithi037 ceph-mon[29204]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:47.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:47 smithi069 ceph-mon[32534]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:48 smithi037 ceph-mon[29204]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:48 smithi069 ceph-mon[32534]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:48 smithi044 ceph-mon[32590]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:50 smithi044 ceph-mon[32590]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:50 smithi037 ceph-mon[29204]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:50 smithi069 ceph-mon[32534]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:51.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.248+0000 7fd3e37fe700 1 -- 172.21.15.44:0/1058289507 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd3f4062e80 con 0x7fd3dc0a3c40 2024-04-25T13:22:51.249 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 10 on host 'smithi044' 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 msgr2=0x7fd3dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4005fb0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 msgr2=0x7fd3f41a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec007ee0 tx=0x7fd3ec00d3b0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 shutdown_connections 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd3dc0a3c40 0x7fd3dc0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3f4104640 0x7fd3f41a0190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3f4102aa0 0x7fd3f4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 --2- 172.21.15.44:0/1058289507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3f4103ca0 0x7fd3f41991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 >> 172.21.15.44:0/1058289507 conn(0x7fd3f40fe040 msgr2=0x7fd3f4107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 shutdown_connections 2024-04-25T13:22:51.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-25T13:22:51.251+0000 7fd3f8a64700 1 -- 172.21.15.44:0/1058289507 wait complete. 2024-04-25T13:22:51.842 DEBUG:teuthology.orchestra.run.smithi044:osd.10> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T13:22:51.845 INFO:tasks.cephadm:Deploying osd.11 on smithi069 with /dev/vg_nvme/lv_1... 2024-04-25T13:22:51.845 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T13:22:52.092 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.529 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:52 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.540 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:52 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:52.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:52 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:53.471 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:53 smithi044 ceph-mon[32590]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:53.562 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:53 smithi069 ceph-mon[32534]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: from='osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:53 smithi037 ceph-mon[29204]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112726 s, 930 MB/s 2024-04-25T13:22:54.101 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-04-25T13:22:54.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-mon[32590]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:22:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-mon[32590]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T13:22:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-mon[32590]: from='osd.10 ' entity='osd.10' 2024-04-25T13:22:54.390 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:22:54.130+0000 7fc29be97700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T13:22:54.390 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:22:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:22:54.204+0000 7fc293481700 -1 osd.10 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:22:54.473 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:54 smithi069 ceph-mon[32534]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:22:54.473 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:54 smithi069 ceph-mon[32534]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T13:22:54.473 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.473 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:54 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.473 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:54 smithi069 ceph-mon[32534]: from='osd.10 ' entity='osd.10' 2024-04-25T13:22:54.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:54 smithi037 ceph-mon[29204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-04-25T13:22:54.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:54 smithi037 ceph-mon[29204]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T13:22:54.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:54 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:54.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:54 smithi037 ceph-mon[29204]: from='osd.10 ' entity='osd.10' 2024-04-25T13:22:54.867 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-04-25T13:22:55.102 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:22:55.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:22:55.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:22:55.354 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:55.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] boot 2024-04-25T13:22:55.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:22:55.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.355 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:55 smithi044 ceph-mon[32590]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:22:55.551 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:22:55.551 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] boot 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.552 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:55 smithi069 ceph-mon[32534]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 894 GiB / 894 GiB avail 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: osd.10 [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] boot 2024-04-25T13:22:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T13:22:55.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:22:55.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:55 smithi037 ceph-mon[29204]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi044 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.531 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.532 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:56 smithi069 ceph-mon[32534]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:22:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 -- 172.21.15.69:0/2693631567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c102a90 msgr2=0x7f798c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 --2- 172.21.15.69:0/2693631567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c102a90 0x7f798c102eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f797c009990 tx=0x7f797c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 -- 172.21.15.69:0/2693631567 shutdown_connections 2024-04-25T13:22:56.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 --2- 172.21.15.69:0/2693631567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 0x7f798c10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 --2- 172.21.15.69:0/2693631567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c102a90 0x7f798c102eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.735+0000 7f7991e12700 1 --2- 172.21.15.69:0/2693631567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c103c90 0x7f798c1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- 172.21.15.69:0/2693631567 >> 172.21.15.69:0/2693631567 conn(0x7f798c0fe030 msgr2=0x7f798c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- 172.21.15.69:0/2693631567 shutdown_connections 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- 172.21.15.69:0/2693631567 wait complete. 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 Processor -- start 2024-04-25T13:22:56.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- start start 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102a90 0x7f798c076790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 0x7f798c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e600 con 0x7f798c102a90 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e480 con 0x7f798c104630 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.736+0000 7f7991e12700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f798c10e780 con 0x7f798c103c90 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59600/0 (socket says 172.21.15.69:59600) 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 -- 172.21.15.69:0/1054989856 learned_addr learned my addr 172.21.15.69:0/1054989856 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798bfff700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 0x7f798c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:56.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798b7fe700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102a90 0x7f798c076790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 -- 172.21.15.69:0/1054989856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 msgr2=0x7f798c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 0x7f798c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 -- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102a90 msgr2=0x7f798c076790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102a90 0x7f798c076790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 -- 172.21.15.69:0/1054989856 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c075c40 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f798affd700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f798000b530 tx=0x7f798000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980015020 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980011ce0 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c1a2e20 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980010e40 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.737+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1a32a0 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.738+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c02df80 con 0x7f798c103c90 2024-04-25T13:22:56.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.739+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7980018020 con 0x7f798c103c90 2024-04-25T13:22:56.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.740+0000 7f7988ff9700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 0x7f79740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:22:56.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.740+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6723+0+0 (secure 0 0 0) 0x7f79800cf680 con 0x7f798c103c90 2024-04-25T13:22:56.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.740+0000 7f798b7fe700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 0x7f79740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:22:56.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.741+0000 7f798b7fe700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 0x7f79740a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f797c0038a0 tx=0x7f797c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:22:56.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.744+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798001f020 con 0x7f798c103c90 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi044 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:56 smithi037 ceph-mon[29204]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi044 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 4273M 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:22:56.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:56 smithi044 ceph-mon[32590]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T13:22:56.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:22:56.955+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f798c062e80 con 0x7f79740a3c40 2024-04-25T13:22:57.695 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:57 smithi069 ceph-mon[32534]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:22:57.695 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:57.695 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:57.695 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:57 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:57.695 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:57 smithi069 ceph-mon[32534]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:22:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:57 smithi037 ceph-mon[29204]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:22:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:57 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:57 smithi037 ceph-mon[29204]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:22:57.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:57 smithi044 ceph-mon[32590]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:22:57.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T13:22:57.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T13:22:57.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:57 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:22:57.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:57 smithi044 ceph-mon[32590]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T13:22:58.683 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:58 smithi069 ceph-mon[32534]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:58.683 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:58 smithi069 ceph-mon[32534]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T13:22:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:58 smithi037 ceph-mon[29204]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:58 smithi037 ceph-mon[29204]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T13:22:58.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:58 smithi044 ceph-mon[32590]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:22:58.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:58 smithi044 ceph-mon[32590]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T13:22:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:22:59 smithi037 ceph-mon[29204]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:22:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:22:59 smithi069 ceph-mon[32534]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:22:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:22:59 smithi044 ceph-mon[32590]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/3295656184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]': finished 2024-04-25T13:23:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:23:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:00.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.69:0/1877968019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/3295656184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]': finished 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:00 smithi069 ceph-mon[32534]: from='client.? 172.21.15.69:0/1877968019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:23:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/3295656184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]: dispatch 2024-04-25T13:23:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "359839d6-6c86-46ea-be27-6f5d8fbcdece"}]': finished 2024-04-25T13:23:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T13:23:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:00 smithi044 ceph-mon[32590]: from='client.? 172.21.15.69:0/1877968019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T13:23:01.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:01 smithi037 ceph-mon[29204]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:01 smithi069 ceph-mon[32534]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:01 smithi044 ceph-mon[32590]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 357 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:03.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:03 smithi037 ceph-mon[29204]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:23:03.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:03 smithi069 ceph-mon[32534]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:23:03.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:03 smithi044 ceph-mon[32590]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T13:23:05.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:05 smithi069 ceph-mon[32534]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:23:05.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:05 smithi037 ceph-mon[29204]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:23:05.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:05 smithi044 ceph-mon[32590]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T13:23:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:07 smithi069 ceph-mon[32534]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:23:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:23:07.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:07 smithi037 ceph-mon[29204]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:23:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:23:07.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:07 smithi044 ceph-mon[32590]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T13:23:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T13:23:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:08.618 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:08 smithi069 ceph-mon[32534]: Deploying daemon osd.11 on smithi069 2024-04-25T13:23:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:08 smithi037 ceph-mon[29204]: Deploying daemon osd.11 on smithi069 2024-04-25T13:23:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:08 smithi044 ceph-mon[32590]: Deploying daemon osd.11 on smithi069 2024-04-25T13:23:09.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:09 smithi037 ceph-mon[29204]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:23:09.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:09 smithi069 ceph-mon[32534]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:23:09.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:09 smithi044 ceph-mon[32590]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T13:23:11.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:11 smithi069 ceph-mon[32534]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:23:11.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:11 smithi037 ceph-mon[29204]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:23:11.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:11 smithi044 ceph-mon[32590]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-25T13:23:13.672 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:13 smithi069 ceph-mon[32534]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:23:13.672 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:13 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.672 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:13 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.672 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:13 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:13.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:13 smithi037 ceph-mon[29204]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:23:13.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:13 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:13 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:13 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:13 smithi044 ceph-mon[32590]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T13:23:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:13 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:13 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:13 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:15.566 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:15 smithi069 ceph-mon[32534]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:15.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:15 smithi037 ceph-mon[29204]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:15.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:15 smithi044 ceph-mon[32590]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:17.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.077+0000 7f7988ff9700 1 -- 172.21.15.69:0/1054989856 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f798c062e80 con 0x7f79740a3c40 2024-04-25T13:23:17.078 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 11 on host 'smithi069' 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 msgr2=0x7f79740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 0x7f79740a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f797c0038a0 tx=0x7f797c005fd0 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 msgr2=0x7f798c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f798000b530 tx=0x7f798000b840 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 shutdown_connections 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f79740a3c40 0x7f79740a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f798c104630 0x7f798c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f798c102a90 0x7f798c076790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 --2- 172.21.15.69:0/1054989856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f798c103c90 0x7f798c074e30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 >> 172.21.15.69:0/1054989856 conn(0x7f798c0fe030 msgr2=0x7f798c1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:17.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 shutdown_connections 2024-04-25T13:23:17.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:17.079+0000 7f7991e12700 1 -- 172.21.15.69:0/1054989856 wait complete. 2024-04-25T13:23:17.434 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:17.434 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.435 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:17 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:17 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:17.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:17 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:17.735 DEBUG:teuthology.orchestra.run.smithi069:osd.11> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T13:23:17.737 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T13:23:17.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:23:17.996 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:18.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:18 smithi069 ceph-mon[32534]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:23:18.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:18 smithi069 ceph-mon[32534]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:23:18.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:18 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:18.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:18 smithi069 ceph-mon[32534]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:18.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:18 smithi069 ceph-mon[32534]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:18 smithi037 ceph-mon[29204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:23:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:18 smithi037 ceph-mon[29204]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:23:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:18 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:18.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:18 smithi037 ceph-mon[29204]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:18.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:18 smithi037 ceph-mon[29204]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:18.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:18 smithi044 ceph-mon[32590]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T13:23:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:18 smithi044 ceph-mon[32590]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T13:23:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:18 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:18 smithi044 ceph-mon[32590]: from='osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:18 smithi044 ceph-mon[32590]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-04-25T13:23:19.385 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:23:19.345+0000 7fe173de9700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.558 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:19 smithi037 ceph-mon[29204]: from='osd.11 ' entity='osd.11' 2024-04-25T13:23:19.684 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:23:19.515+0000 7fe16bbd4700 -1 osd.11 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.685 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:19 smithi069 ceph-mon[32534]: from='osd.11 ' entity='osd.11' 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 -- 172.21.15.37:0/2697203537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150103540 msgr2=0x7f7150109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 --2- 172.21.15.37:0/2697203537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150103540 0x7f7150109df0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f714800b0d0 tx=0x7f714800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 -- 172.21.15.37:0/2697203537 shutdown_connections 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 --2- 172.21.15.37:0/2697203537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71501019a0 0x7f7150101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 --2- 172.21.15.37:0/2697203537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150103540 0x7f7150109df0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 --2- 172.21.15.37:0/2697203537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7150102ba0 0x7f7150103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 -- 172.21.15.37:0/2697203537 >> 172.21.15.37:0/2697203537 conn(0x7f71500fcf40 msgr2=0x7f71500ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 -- 172.21.15.37:0/2697203537 shutdown_connections 2024-04-25T13:23:19.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.767+0000 7f7154cec700 1 -- 172.21.15.37:0/2697203537 wait complete. 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 Processor -- start 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 -- start start 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71501019a0 0x7f715006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7150103540 0x7f715019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:19.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f715010d440 con 0x7f7150102ba0 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f715010d2c0 con 0x7f7150103540 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f7154cec700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f715010d5c0 con 0x7f71501019a0 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f714dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f714dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50948/0 (socket says 172.21.15.37:50948) 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f714dd9b700 1 -- 172.21.15.37:0/1145047653 learned_addr learned my addr 172.21.15.37:0/1145047653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f714e59c700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71501019a0 0x7f715006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.768+0000 7f714ed9d700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7150103540 0x7f715019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 -- 172.21.15.37:0/1145047653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71501019a0 msgr2=0x7f715006b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71501019a0 0x7f715006b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 -- 172.21.15.37:0/1145047653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7150103540 msgr2=0x7f715019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7150103540 0x7f715019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 -- 172.21.15.37:0/1145047653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71501a0790 con 0x7f7150102ba0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f714dd9b700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f713800e9c0 tx=0x7f713800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713800cbd0 con 0x7f7150102ba0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f713800ee60 con 0x7f7150102ba0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71501a0a80 con 0x7f7150102ba0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71380055c0 con 0x7f7150102ba0 2024-04-25T13:23:19.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.769+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501a0fc0 con 0x7f7150102ba0 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.770+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7150069730 con 0x7f7150102ba0 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.771+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7138004750 con 0x7f7150102ba0 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.772+0000 7f71477fe700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 0x7f713c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.772+0000 7f714e59c700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 0x7f713c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.772+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 6847+0+0 (secure 0 0 0) 0x7f71380d00b0 con 0x7f7150102ba0 2024-04-25T13:23:19.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.772+0000 7f714e59c700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 0x7f713c0a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7150102a00 tx=0x7f714000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.777+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f713809cfb0 con 0x7f7150102ba0 2024-04-25T13:23:19.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:19.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-04-25T13:23:19.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T13:23:19.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:19.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:19 smithi044 ceph-mon[32590]: from='osd.11 ' entity='osd.11' 2024-04-25T13:23:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.984+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7150063650 con 0x7f7150102ba0 2024-04-25T13:23:19.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.985+0000 7f71477fe700 1 -- 172.21.15.37:0/1145047653 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7f713809c8b0 con 0x7f7150102ba0 2024-04-25T13:23:19.985 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:19.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 msgr2=0x7f713c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:19.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 0x7f713c0a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7150102a00 tx=0x7f714000a3b0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 msgr2=0x7f715019d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f713800e9c0 tx=0x7f713800ecd0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 shutdown_connections 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f713c0a3cc0 0x7f713c0a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7150103540 0x7f715019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71501019a0 0x7f715006b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 --2- 172.21.15.37:0/1145047653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7150102ba0 0x7f715019d870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 >> 172.21.15.37:0/1145047653 conn(0x7f71500fcf40 msgr2=0x7f7150105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 shutdown_connections 2024-04-25T13:23:19.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:19.987+0000 7f7154cec700 1 -- 172.21.15.37:0/1145047653 wait complete. 2024-04-25T13:23:20.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:20 smithi037 ceph-mon[29204]: purged_snaps scrub starts 2024-04-25T13:23:20.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:20 smithi037 ceph-mon[29204]: purged_snaps scrub ok 2024-04-25T13:23:20.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:20 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:20.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:20 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1145047653' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:20.828 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"num_osds":12,"num_up_osds":11,"osd_up_since":1714051374,"num_in_osds":12,"osd_in_since":1714051379,"num_remapped_pgs":0} 2024-04-25T13:23:20.837 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:20 smithi069 ceph-mon[32534]: purged_snaps scrub starts 2024-04-25T13:23:20.838 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:20 smithi069 ceph-mon[32534]: purged_snaps scrub ok 2024-04-25T13:23:20.838 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:20 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:20.838 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:20 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1145047653' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:20 smithi044 ceph-mon[32590]: purged_snaps scrub starts 2024-04-25T13:23:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:20 smithi044 ceph-mon[32590]: purged_snaps scrub ok 2024-04-25T13:23:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:20 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:20 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1145047653' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] boot 2024-04-25T13:23:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:23:21.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:21 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.829 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T13:23:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] boot 2024-04-25T13:23:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:23:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:21 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 983 GiB / 983 GiB avail 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: osd.11 [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] boot 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:21.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:21.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:21 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:22.084 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:22 smithi037 ceph-mon[29204]: Detected new or changed devices on smithi069 2024-04-25T13:23:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:22 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T13:23:22.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:22 smithi037 ceph-mon[29204]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:23:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:22 smithi069 ceph-mon[32534]: Detected new or changed devices on smithi069 2024-04-25T13:23:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:22 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T13:23:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:22 smithi069 ceph-mon[32534]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:23:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:22 smithi044 ceph-mon[32590]: Detected new or changed devices on smithi069 2024-04-25T13:23:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:22 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 4272M 2024-04-25T13:23:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:22 smithi044 ceph-mon[32590]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T13:23:23.787 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:23 smithi037 ceph-mon[29204]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:23.790 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:23 smithi044 ceph-mon[32590]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.822+0000 7fc406d0c700 1 -- 172.21.15.37:0/2678931708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 msgr2=0x7fc400103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 --2- 172.21.15.37:0/2678931708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0099c0 tx=0x7fc3ec009cd0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 -- 172.21.15.37:0/2678931708 shutdown_connections 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 --2- 172.21.15.37:0/2678931708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc400103560 0x7fc400109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 --2- 172.21.15.37:0/2678931708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4001019c0 0x7fc400101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 --2- 172.21.15.37:0/2678931708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 -- 172.21.15.37:0/2678931708 >> 172.21.15.37:0/2678931708 conn(0x7fc4000fcf60 msgr2=0x7fc4000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 -- 172.21.15.37:0/2678931708 shutdown_connections 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 -- 172.21.15.37:0/2678931708 wait complete. 2024-04-25T13:23:23.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 Processor -- start 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 -- start start 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.823+0000 7fc406d0c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4001019c0 0x7fc4001058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 0x7fc4001044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc40010d430 con 0x7fc400102bc0 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc40010d2b0 con 0x7fc4001019c0 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc40010d5b0 con 0x7fc400103560 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50968/0 (socket says 172.21.15.37:50968) 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 -- 172.21.15.37:0/1778543250 learned_addr learned my addr 172.21.15.37:0/1778543250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc404aa8700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4001019c0 0x7fc4001058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc4052a9700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 0x7fc4001044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:23.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 -- 172.21.15.37:0/1778543250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 msgr2=0x7fc4001044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 0x7fc4001044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 -- 172.21.15.37:0/1778543250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4001019c0 msgr2=0x7fc4001058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4001019c0 0x7fc4001058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 -- 172.21.15.37:0/1778543250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc400104d20 con 0x7fc400102bc0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc4052a9700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 0x7fc4001044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fffff700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005910 tx=0x7fc3ec00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec027070 con 0x7fc400102bc0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec022920 con 0x7fc400102bc0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4001a2cb0 con 0x7fc400102bc0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.824+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec009e60 con 0x7fc400102bc0 2024-04-25T13:23:23.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.825+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4001a3100 con 0x7fc400102bc0 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.826+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4001a32d0 con 0x7fc400102bc0 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.827+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3ec02a070 con 0x7fc400102bc0 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.827+0000 7fc3fdffb700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 0x7fc3f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.827+0000 7fc404aa8700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 0x7fc3f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.827+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fc3ec0d1050 con 0x7fc400102bc0 2024-04-25T13:23:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.828+0000 7fc404aa8700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 0x7fc3f00a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc400102a20 tx=0x7fc3f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:23.831+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3ec013070 con 0x7fc400102bc0 2024-04-25T13:23:23.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:23 smithi069 ceph-mon[32534]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.038+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc4001a3660 con 0x7fc400102bc0 2024-04-25T13:23:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.039+0000 7fc3fdffb700 1 -- 172.21.15.37:0/1778543250 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v85) ==== 74+0+133 (secure 0 0 0) 0x7fc3ec09df60 con 0x7fc400102bc0 2024-04-25T13:23:24.039 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 msgr2=0x7fc3f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 0x7fc3f00a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc400102a20 tx=0x7fc3f4009260 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 msgr2=0x7fc400103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec005910 tx=0x7fc3ec00fa00 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 shutdown_connections 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3f00a3c40 0x7fc3f00a6100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4001019c0 0x7fc4001058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc400103560 0x7fc4001044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 --2- 172.21.15.37:0/1778543250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc400102bc0 0x7fc400103fa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:24.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 >> 172.21.15.37:0/1778543250 conn(0x7fc4000fcf60 msgr2=0x7fc4000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 shutdown_connections 2024-04-25T13:23:24.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:24.041+0000 7fc406d0c700 1 -- 172.21.15.37:0/1778543250 wait complete. 2024-04-25T13:23:24.591 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:24 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1778543250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:24.620 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"num_osds":12,"num_up_osds":12,"osd_up_since":1714051400,"num_in_osds":12,"osd_in_since":1714051379,"num_remapped_pgs":0} 2024-04-25T13:23:24.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:23:24.807 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:24 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1778543250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:24.868 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:24 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1778543250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T13:23:25.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:25 smithi037 ceph-mon[29204]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:25 smithi069 ceph-mon[32534]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:25.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:25 smithi044 ceph-mon[32590]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 -- 172.21.15.37:0/629765954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228107c30 msgr2=0x7fd228108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 --2- 172.21.15.37:0/629765954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228107c30 0x7fd228108010 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099c0 tx=0x7fd218009cd0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 -- 172.21.15.37:0/629765954 shutdown_connections 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 --2- 172.21.15.37:0/629765954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228102ed0 0x7fd22810e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 --2- 172.21.15.37:0/629765954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228107c30 0x7fd228108010 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 --2- 172.21.15.37:0/629765954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102510 0x7fd228102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 -- 172.21.15.37:0/629765954 >> 172.21.15.37:0/629765954 conn(0x7fd228076c60 msgr2=0x7fd228077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:26.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 -- 172.21.15.37:0/629765954 shutdown_connections 2024-04-25T13:23:26.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.392+0000 7fd22d71c700 1 -- 172.21.15.37:0/629765954 wait complete. 2024-04-25T13:23:26.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 Processor -- start 2024-04-25T13:23:26.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 -- start start 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102ed0 0x7fd22819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 0x7fd22819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111cc0 con 0x7fd228102510 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111b40 con 0x7fd228102ed0 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd22d71c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd228111e40 con 0x7fd228107c30 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.393+0000 7fd226ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:26.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50990/0 (socket says 172.21.15.37:50990) 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 -- 172.21.15.37:0/146934766 learned_addr learned my addr 172.21.15.37:0/146934766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd2267fc700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102ed0 0x7fd22819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd2277fe700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 0x7fd22819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 -- 172.21.15.37:0/146934766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 msgr2=0x7fd22819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 0x7fd22819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 -- 172.21.15.37:0/146934766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102ed0 msgr2=0x7fd22819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102ed0 0x7fd22819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 -- 172.21.15.37:0/146934766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a0450 con 0x7fd228102510 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd2277fe700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 0x7fd22819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd226ffd700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd218005910 tx=0x7fd218004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218026070 con 0x7fd228102510 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a06e0 con 0x7fd228102510 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd218003c70 con 0x7fd228102510 2024-04-25T13:23:26.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21800fb50 con 0x7fd228102510 2024-04-25T13:23:26.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.394+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281a7fc0 con 0x7fd228102510 2024-04-25T13:23:26.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.395+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228069730 con 0x7fd228102510 2024-04-25T13:23:26.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.396+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd218029070 con 0x7fd228102510 2024-04-25T13:23:26.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.397+0000 7fd20ffff700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 0x7fd2100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:26.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.397+0000 7fd2267fc700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 0x7fd2100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:26.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.397+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fd2180d0750 con 0x7fd228102510 2024-04-25T13:23:26.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.397+0000 7fd2267fc700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 0x7fd2100a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2280771f0 tx=0x7fd21c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:26.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.402+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd21805dff0 con 0x7fd228102510 2024-04-25T13:23:26.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.610+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd22802df80 con 0x7fd228102510 2024-04-25T13:23:26.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.611+0000 7fd20ffff700 1 -- 172.21.15.37:0/146934766 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18373 (secure 0 0 0) 0x7fd218013070 con 0x7fd228102510 2024-04-25T13:23:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:26.612 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:23:21.894777+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 msgr2=0x7fd2100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 0x7fd2100a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd2280771f0 tx=0x7fd21c009260 comp rx=0 tx=0).stop 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 msgr2=0x7fd22819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd218005910 tx=0x7fd218004270 comp rx=0 tx=0).stop 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 shutdown_connections 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd2100a3cc0 0x7fd2100a6180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd228107c30 0x7fd22819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd228102510 0x7fd22819b440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 --2- 172.21.15.37:0/146934766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd228102ed0 0x7fd22819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 >> 172.21.15.37:0/146934766 conn(0x7fd228076c60 msgr2=0x7fd22810ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 shutdown_connections 2024-04-25T13:23:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:26.613+0000 7fd22d71c700 1 -- 172.21.15.37:0/146934766 wait complete. 2024-04-25T13:23:27.236 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:23:27.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T13:23:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:23:27.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:27 smithi037 ceph-mon[29204]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:27.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:27 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/146934766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:27.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:27 smithi069 ceph-mon[32534]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:27.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:27 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/146934766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:27 smithi044 ceph-mon[32590]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:27 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/146934766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 -- 172.21.15.37:0/674903204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406afe0 msgr2=0x7f6a3406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 --2- 172.21.15.37:0/674903204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406afe0 0x7f6a3406b440 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009990 tx=0x7f6a28009ca0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 -- 172.21.15.37:0/674903204 shutdown_connections 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 --2- 172.21.15.37:0/674903204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406b980 0x7f6a3410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 --2- 172.21.15.37:0/674903204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 --2- 172.21.15.37:0/674903204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406afe0 0x7f6a3406b440 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 -- 172.21.15.37:0/674903204 >> 172.21.15.37:0/674903204 conn(0x7f6a340788b0 msgr2=0x7f6a34078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 -- 172.21.15.37:0/674903204 shutdown_connections 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.144+0000 7f6a39946700 1 -- 172.21.15.37:0/674903204 wait complete. 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 Processor -- start 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 -- start start 2024-04-25T13:23:29.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406afe0 0x7f6a34101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34112be0 con 0x7f6a3406b980 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34112a60 con 0x7f6a3406afe0 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.145+0000 7f6a39946700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34112d60 con 0x7f6a34108a20 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51002/0 (socket says 172.21.15.37:51002) 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 -- 172.21.15.37:0/757690261 learned_addr learned my addr 172.21.15.37:0/757690261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a337fe700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a32ffd700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406afe0 0x7f6a34101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 -- 172.21.15.37:0/757690261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 msgr2=0x7f6a34102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 -- 172.21.15.37:0/757690261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406afe0 msgr2=0x7f6a34101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:29.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406afe0 0x7f6a34101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 -- 172.21.15.37:0/757690261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341a7640 con 0x7f6a3406b980 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a337fe700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a327fc700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a28000c00 tx=0x7f6a28009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28027070 con 0x7f6a3406b980 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341a78d0 con 0x7f6a3406b980 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28022470 con 0x7f6a3406b980 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a28017650 con 0x7f6a3406b980 2024-04-25T13:23:29.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.146+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a7d20 con 0x7f6a3406b980 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.147+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a34069730 con 0x7f6a3406b980 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.149+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a2802a070 con 0x7f6a3406b980 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.149+0000 7f6a38944700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 0x7f6a1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.149+0000 7f6a32ffd700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 0x7f6a1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.149+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f6a280d0e90 con 0x7f6a3406b980 2024-04-25T13:23:29.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.150+0000 7f6a32ffd700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 0x7f6a1c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009be0 tx=0x7f6a24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:29.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.153+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a28013070 con 0x7f6a3406b980 2024-04-25T13:23:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.369+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a3402e140 con 0x7f6a3406b980 2024-04-25T13:23:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.369+0000 7f6a38944700 1 -- 172.21.15.37:0/757690261 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f6a2809dda0 con 0x7f6a3406b980 2024-04-25T13:23:29.370 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 msgr2=0x7f6a1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 0x7f6a1c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009be0 tx=0x7f6a24009260 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 msgr2=0x7f6a34101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a28000c00 tx=0x7f6a28009580 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 shutdown_connections 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a1c0a3c40 0x7f6a1c0a6100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3406afe0 0x7f6a34101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a34108a20 0x7f6a34102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 --2- 172.21.15.37:0/757690261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3406b980 0x7f6a34101f40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 >> 172.21.15.37:0/757690261 conn(0x7f6a340788b0 msgr2=0x7f6a340ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 shutdown_connections 2024-04-25T13:23:29.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:29.372+0000 7f6a39946700 1 -- 172.21.15.37:0/757690261 wait complete. 2024-04-25T13:23:29.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:29 smithi037 ceph-mon[29204]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:29.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:29 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/757690261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:29 smithi069 ceph-mon[32534]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:29 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/757690261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:29.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:29 smithi044 ceph-mon[32590]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:29.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:29 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/757690261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.b on smithi044 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.e on smithi044 2024-04-25T13:23:30.166 INFO:tasks.cephadm:Adding mds.h on smithi044 2024-04-25T13:23:30.167 INFO:tasks.cephadm:Adding mds.k on smithi044 2024-04-25T13:23:30.167 INFO:tasks.cephadm:Adding mds.c on smithi069 2024-04-25T13:23:30.167 INFO:tasks.cephadm:Adding mds.f on smithi069 2024-04-25T13:23:30.167 INFO:tasks.cephadm:Adding mds.i on smithi069 2024-04-25T13:23:30.167 INFO:tasks.cephadm:Adding mds.l on smithi069 2024-04-25T13:23:30.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l' 2024-04-25T13:23:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:23:31.784 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:31 smithi069 ceph-mon[32534]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:31.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:31 smithi037 ceph-mon[29204]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:31.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:31 smithi044 ceph-mon[32590]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:32.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 -- 172.21.15.69:0/1282446698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4784105e80 msgr2=0x7f4784108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:32.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 --2- 172.21.15.69:0/1282446698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4784105e80 0x7f4784108270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4774009990 tx=0x7f4774009ca0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 -- 172.21.15.69:0/1282446698 shutdown_connections 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 --2- 172.21.15.69:0/1282446698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47841087b0 0x7f478410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 --2- 172.21.15.69:0/1282446698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784103550 0x7f4784105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 --2- 172.21.15.69:0/1282446698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4784105e80 0x7f4784108270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 -- 172.21.15.69:0/1282446698 >> 172.21.15.69:0/1282446698 conn(0x7f47840fcfa0 msgr2=0x7f47840ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 -- 172.21.15.69:0/1282446698 shutdown_connections 2024-04-25T13:23:32.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 -- 172.21.15.69:0/1282446698 wait complete. 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.024+0000 7f478c676700 1 Processor -- start 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 -- start start 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4784103550 0x7f4784198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784105e80 0x7f47841992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f478410e640 con 0x7f4784105e80 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f478410e4c0 con 0x7f4784103550 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f478410e7c0 con 0x7f47841087b0 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46230/0 (socket says 172.21.15.69:46230) 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 -- 172.21.15.69:0/2423624130 learned_addr learned my addr 172.21.15.69:0/2423624130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-04-25T13:23:32.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f4789c11700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784105e80 0x7f47841992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478a412700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4784103550 0x7f4784198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 -- 172.21.15.69:0/2423624130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4784103550 msgr2=0x7f4784198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4784103550 0x7f4784198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 -- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784105e80 msgr2=0x7f47841992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784105e80 0x7f47841992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 -- 172.21.15.69:0/2423624130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47841a09c0 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478ac13700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f477c00ba10 tx=0x7f477c00bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c019070 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c00cde0 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47841a0cb0 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.025+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c012340 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.026+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841a1260 con 0x7f47841087b0 2024-04-25T13:23:32.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.026+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784069730 con 0x7f47841087b0 2024-04-25T13:23:32.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.027+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f477c01c070 con 0x7f47841087b0 2024-04-25T13:23:32.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.028+0000 7f477b7fe700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 0x7f47700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:32.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.028+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f477c0cfa30 con 0x7f47841087b0 2024-04-25T13:23:32.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.028+0000 7f478a412700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 0x7f47700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:32.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.029+0000 7f478a412700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 0x7f47700a6180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:32.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.032+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f477c023070 con 0x7f47841087b0 2024-04-25T13:23:32.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.246+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l", "target": ["mon-mgr", ""]}) -- 0x7f4784063140 con 0x7f47700a3cc0 2024-04-25T13:23:32.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.255+0000 7f477b7fe700 1 -- 172.21.15.69:0/2423624130 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4784063140 con 0x7f47700a3cc0 2024-04-25T13:23:32.256 INFO:teuthology.orchestra.run.smithi069.stdout:Scheduled mds.all update... 2024-04-25T13:23:32.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.257+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 msgr2=0x7f47700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:32.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.257+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 0x7f47700a6180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780008040 comp rx=0 tx=0).stop 2024-04-25T13:23:32.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.257+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 msgr2=0x7f47841a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.257+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f477c00ba10 tx=0x7f477c00bd20 comp rx=0 tx=0).stop 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 shutdown_connections 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f47700a3cc0 0x7f47700a6180 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4784103550 0x7f4784198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47841087b0 0x7f47841a0320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 --2- 172.21.15.69:0/2423624130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4784105e80 0x7f47841992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 >> 172.21.15.69:0/2423624130 conn(0x7f47840fcfa0 msgr2=0x7f4784103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 shutdown_connections 2024-04-25T13:23:32.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-04-25T13:23:32.258+0000 7f478c676700 1 -- 172.21.15.69:0/2423624130 wait complete. 2024-04-25T13:23:32.929 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T13:23:32.932 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T13:23:32.975 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T13:23:33.018 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T13:23:33.061 DEBUG:teuthology.orchestra.run.smithi044:mds.b> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T13:23:33.063 DEBUG:teuthology.orchestra.run.smithi044:mds.e> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T13:23:33.106 DEBUG:teuthology.orchestra.run.smithi044:mds.h> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T13:23:33.149 DEBUG:teuthology.orchestra.run.smithi044:mds.k> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T13:23:33.192 DEBUG:teuthology.orchestra.run.smithi069:mds.c> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T13:23:33.194 DEBUG:teuthology.orchestra.run.smithi069:mds.f> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T13:23:33.237 DEBUG:teuthology.orchestra.run.smithi069:mds.i> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T13:23:33.282 DEBUG:teuthology.orchestra.run.smithi069:mds.l> sudo journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T13:23:33.325 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T13:23:33.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l;count:12 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:33.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:33 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 -- 172.21.15.37:0/3235804478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9880101fb0 msgr2=0x7f98801023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 --2- 172.21.15.37:0/3235804478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9880101fb0 0x7f98801023d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9870009900 tx=0x7f987002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 -- 172.21.15.37:0/3235804478 shutdown_connections 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 --2- 172.21.15.37:0/3235804478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880103b60 0x7f988010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 --2- 172.21.15.37:0/3235804478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98801031a0 0x7f9880103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 --2- 172.21.15.37:0/3235804478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9880101fb0 0x7f98801023d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 -- 172.21.15.37:0/3235804478 >> 172.21.15.37:0/3235804478 conn(0x7f98800fd540 msgr2=0x7f98800ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 -- 172.21.15.37:0/3235804478 shutdown_connections 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 -- 172.21.15.37:0/3235804478 wait complete. 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.445+0000 7f98860bb640 1 Processor -- start 2024-04-25T13:23:33.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 -- start start 2024-04-25T13:23:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 0x7f98801981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9880103b60 0x7f988019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f988010daf0 con 0x7f98801031a0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f988010d970 con 0x7f9880103b60 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f98860bb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f988010dc70 con 0x7f9880101fb0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f987f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 0x7f98801981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42978/0 (socket says 172.21.15.37:42978) 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 -- 172.21.15.37:0/245841199 learned_addr learned my addr 172.21.15.37:0/245841199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f987ffff640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9880103b60 0x7f988019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 -- 172.21.15.37:0/245841199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 msgr2=0x7f98801981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 0x7f98801981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 -- 172.21.15.37:0/245841199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9880103b60 msgr2=0x7f988019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9880103b60 0x7f988019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 -- 172.21.15.37:0/245841199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988019fdc0 con 0x7f98801031a0 2024-04-25T13:23:33.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f987f7fe640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 0x7f98801981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.446+0000 7f9876ffd640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f986800c920 tx=0x7f986800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.447+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868007c70 con 0x7f98801031a0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.447+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98801a00b0 con 0x7f98801031a0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.447+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9868007e10 con 0x7f98801031a0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.447+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868018820 con 0x7f98801031a0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.447+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98801a0650 con 0x7f98801031a0 2024-04-25T13:23:33.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.448+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98801092d0 con 0x7f98801031a0 2024-04-25T13:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.449+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f986800f450 con 0x7f98801031a0 2024-04-25T13:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.450+0000 7f987d7fa640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 0x7f986c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.450+0000 7f987f7fe640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 0x7f986c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.450+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f98680d0070 con 0x7f98801031a0 2024-04-25T13:23:33.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.450+0000 7f987f7fe640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 0x7f986c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9870009230 tx=0x7f98700314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:33.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.454+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9868014030 con 0x7f98801031a0 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l;count:12 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:33.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:33.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:33.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:33 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l;count:12 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:23:33.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:33.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:33.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:33.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:33 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:33.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.641+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f988002d7c0 con 0x7f98801031a0 2024-04-25T13:23:33.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.642+0000 7f987d7fa640 1 -- 172.21.15.37:0/245841199 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18373 (secure 0 0 0) 0x7f986809cef0 con 0x7f98801031a0 2024-04-25T13:23:33.643 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:33.643 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:23:21.894777+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 msgr2=0x7f986c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 0x7f986c0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9870009230 tx=0x7f98700314a0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 msgr2=0x7f98801986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f986800c920 tx=0x7f986800cde0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 shutdown_connections 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9880103b60 0x7f988019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f986c0a3bb0 0x7f986c0a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98801031a0 0x7f98801986e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 --2- 172.21.15.37:0/245841199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9880101fb0 0x7f98801981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 >> 172.21.15.37:0/245841199 conn(0x7f98800fd540 msgr2=0x7f98801064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 shutdown_connections 2024-04-25T13:23:33.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.644+0000 7f98860bb640 1 -- 172.21.15.37:0/245841199 wait complete. 2024-04-25T13:23:33.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:23:33.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:23:33.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 -- 172.21.15.37:0/1765134446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084105350 msgr2=0x7ff084107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/1765134446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084105350 0x7ff084107740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff074009900 tx=0x7ff07402a410 comp rx=0 tx=0).stop 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 -- 172.21.15.37:0/1765134446 shutdown_connections 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/1765134446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084107c80 0x7ff08410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/1765134446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084105350 0x7ff084107740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/1765134446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084102a20 0x7ff084104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 -- 172.21.15.37:0/1765134446 >> 172.21.15.37:0/1765134446 conn(0x7ff0840fc430 msgr2=0x7ff0840fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 -- 172.21.15.37:0/1765134446 shutdown_connections 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.761+0000 7ff08bd9b640 1 -- 172.21.15.37:0/1765134446 wait complete. 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 Processor -- start 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 -- start start 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084102a20 0x7ff084198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084105350 0x7ff084198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410daa0 con 0x7ff084107c80 2024-04-25T13:23:33.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410d920 con 0x7ff084102a20 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08bd9b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410dc20 con 0x7ff084105350 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42998/0 (socket says 172.21.15.37:42998) 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 -- 172.21.15.37:0/839548683 learned_addr learned my addr 172.21.15.37:0/839548683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff089336640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084105350 0x7ff084198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff089b37640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084102a20 0x7ff084198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 -- 172.21.15.37:0/839548683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084105350 msgr2=0x7ff084198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084105350 0x7ff084198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 -- 172.21.15.37:0/839548683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084102a20 msgr2=0x7ff084198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084102a20 0x7ff084198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.762+0000 7ff08a338640 1 -- 172.21.15.37:0/839548683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08419fe30 con 0x7ff084107c80 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff08a338640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00cad0 tx=0x7ff07c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00ae20 con 0x7ff084107c80 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07c005020 con 0x7ff084107c80 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0841a0120 con 0x7ff084107c80 2024-04-25T13:23:33.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c00f6b0 con 0x7ff084107c80 2024-04-25T13:23:33.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.763+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a05f0 con 0x7ff084107c80 2024-04-25T13:23:33.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.764+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0840fdf20 con 0x7ff084107c80 2024-04-25T13:23:33.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.765+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff07c0051d0 con 0x7ff084107c80 2024-04-25T13:23:33.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.766+0000 7ff07affd640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 0x7ff0700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:33.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.766+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7ff07c0d0300 con 0x7ff084107c80 2024-04-25T13:23:33.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.766+0000 7ff089b37640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 0x7ff0700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:33.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.766+0000 7ff089b37640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 0x7ff0700a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff080005fd0 tx=0x7ff080005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:33.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.769+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff07c014030 con 0x7ff084107c80 2024-04-25T13:23:33.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.956+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff084062b50 con 0x7ff084107c80 2024-04-25T13:23:33.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.956+0000 7ff07affd640 1 -- 172.21.15.37:0/839548683 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7ff07c09d180 con 0x7ff084107c80 2024-04-25T13:23:33.957 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:23:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.958+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 msgr2=0x7ff0700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.958+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 0x7ff0700a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff080005fd0 tx=0x7ff080005c70 comp rx=0 tx=0).stop 2024-04-25T13:23:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.958+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 msgr2=0x7ff08419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00cad0 tx=0x7ff07c00cf90 comp rx=0 tx=0).stop 2024-04-25T13:23:33.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 shutdown_connections 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0700a3bb0 0x7ff0700a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff084107c80 0x7ff08419f7c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff084105350 0x7ff084198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 --2- 172.21.15.37:0/839548683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff084102a20 0x7ff084198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 >> 172.21.15.37:0/839548683 conn(0x7ff0840fc430 msgr2=0x7ff084103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 shutdown_connections 2024-04-25T13:23:33.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:33.959+0000 7ff08bd9b640 1 -- 172.21.15.37:0/839548683 wait complete. 2024-04-25T13:23:33.970 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- 172.21.15.37:0/3285120995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0941031a0 msgr2=0x7ff094103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3285120995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0941031a0 0x7ff094103620 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- 172.21.15.37:0/3285120995 shutdown_connections 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3285120995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 0x7ff09410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3285120995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0941031a0 0x7ff094103620 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3285120995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff094101fb0 0x7ff0941023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- 172.21.15.37:0/3285120995 >> 172.21.15.37:0/3285120995 conn(0x7ff0940fd540 msgr2=0x7ff0940ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- 172.21.15.37:0/3285120995 shutdown_connections 2024-04-25T13:23:34.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- 172.21.15.37:0/3285120995 wait complete. 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 Processor -- start 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- start start 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0941031a0 0x7ff094198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 0x7ff09419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410daf0 con 0x7ff094101fb0 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410d970 con 0x7ff094103b60 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff09b17d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410dc70 con 0x7ff0941031a0 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.072+0000 7ff098f19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43016/0 (socket says 172.21.15.37:43016) 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 -- 172.21.15.37:0/3312629491 learned_addr learned my addr 172.21.15.37:0/3312629491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff093fff640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0941031a0 0x7ff094198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff09971a640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 0x7ff09419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 -- 172.21.15.37:0/3312629491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0941031a0 msgr2=0x7ff094198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:34.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0941031a0 0x7ff094198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 -- 172.21.15.37:0/3312629491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 msgr2=0x7ff09419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 0x7ff09419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 -- 172.21.15.37:0/3312629491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09419fd60 con 0x7ff094101fb0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff098f19640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9c0 tx=0x7ff08800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08800cd80 con 0x7ff094101fb0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0880035f0 con 0x7ff094101fb0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088010690 con 0x7ff094101fb0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09419fff0 con 0x7ff094101fb0 2024-04-25T13:23:34.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.073+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941a04b0 con 0x7ff094101fb0 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.074+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff094109320 con 0x7ff094101fb0 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.075+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff088004000 con 0x7ff094101fb0 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.076+0000 7ff091ffb640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 0x7ff0840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.076+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7ff0880d04b0 con 0x7ff094101fb0 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.076+0000 7ff093fff640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 0x7ff0840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:34.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.076+0000 7ff093fff640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 0x7ff0840a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff080000c00 tx=0x7ff0800314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:34.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.081+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff08809d330 con 0x7ff094101fb0 2024-04-25T13:23:34.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.307+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff09402d5c0 con 0x7ff094101fb0 2024-04-25T13:23:34.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.307+0000 7ff091ffb640 1 -- 172.21.15.37:0/3312629491 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ff09402d5c0 con 0x7ff094101fb0 2024-04-25T13:23:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:34.308 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T13:23:34.308 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 1 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 msgr2=0x7ff0840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 0x7ff0840a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff080000c00 tx=0x7ff0800314a0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 msgr2=0x7ff094198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff08800e9c0 tx=0x7ff08800ee80 comp rx=0 tx=0).stop 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 shutdown_connections 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff094103b60 0x7ff09419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0941031a0 0x7ff094198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff0840a3c30 0x7ff0840a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 --2- 172.21.15.37:0/3312629491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff094101fb0 0x7ff094198140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 >> 172.21.15.37:0/3312629491 conn(0x7ff0940fd540 msgr2=0x7ff0941064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 shutdown_connections 2024-04-25T13:23:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:34.309+0000 7ff09b17d640 1 -- 172.21.15.37:0/3312629491 wait complete. 2024-04-25T13:23:34.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:34 smithi069 ceph-mon[32534]: Deploying daemon mds.l on smithi069 2024-04-25T13:23:34.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:34 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/245841199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:34.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:34 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/839548683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:34.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:34 smithi037 ceph-mon[29204]: Deploying daemon mds.l on smithi069 2024-04-25T13:23:34.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:34 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/245841199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:34.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:34 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/839548683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:34.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:34 smithi044 ceph-mon[32590]: Deploying daemon mds.l on smithi069 2024-04-25T13:23:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:34 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/245841199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:23:34.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:34 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/839548683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:23:35.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:35 smithi069 ceph-mon[32534]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:35.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:35 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3312629491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:35.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:35 smithi044 ceph-mon[32590]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:35.538 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:35 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3312629491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:35 smithi037 ceph-mon[29204]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:35.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:35 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3312629491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:35.679 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:23:35 smithi069 podman[62171]: 2024-04-25 13:23:35.65655941 +0000 UTC m=+0.355830374 container start fc1337416794d080da6985ded28dd42ea7641b4ea9ee4ad399be6cf675ba8f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:23:36.135 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:23:35 smithi069 ceph-mds[62190]: -- 172.21.15.69:0/394113075 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562057331a40 con 0x562057305000 2024-04-25T13:23:36.135 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:23:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l[62186]: starting mds.l at 2024-04-25T13:23:36.135 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:23:35 smithi069 bash[62171]: fc1337416794d080da6985ded28dd42ea7641b4ea9ee4ad399be6cf675ba8f69 2024-04-25T13:23:36.135 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:23:35 smithi069 systemd[1]: Started Ceph mds.l for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:36.321 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 -- 172.21.15.37:0/710106954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc100e60 msgr2=0x7f22dc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/710106954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc100e60 0x7f22dc101280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009900 tx=0x7f22d802a410 comp rx=0 tx=0).stop 2024-04-25T13:23:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 -- 172.21.15.37:0/710106954 shutdown_connections 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/710106954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc102a00 0x7f22dc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/710106954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 0x7f22dc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/710106954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc100e60 0x7f22dc101280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 -- 172.21.15.37:0/710106954 >> 172.21.15.37:0/710106954 conn(0x7f22dc0fc460 msgr2=0x7f22dc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 -- 172.21.15.37:0/710106954 shutdown_connections 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.432+0000 7f22e3cfa640 1 -- 172.21.15.37:0/710106954 wait complete. 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 Processor -- start 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 -- start start 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 0x7f22dc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 0x7f22dc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10c9d0 con 0x7f22dc102a00 2024-04-25T13:23:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10c850 con 0x7f22dc102040 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e3cfa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10cb50 con 0x7f22dc100e60 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e1a96640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 0x7f22dc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e1295640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 0x7f22dc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43044/0 (socket says 172.21.15.37:43044) 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e1a96640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 0x7f22dc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.37:46360/0 (socket says 172.21.15.37:46360) 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 -- 172.21.15.37:0/4181654142 learned_addr learned my addr 172.21.15.37:0/4181654142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 -- 172.21.15.37:0/4181654142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 msgr2=0x7f22dc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 0x7f22dc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 -- 172.21.15.37:0/4181654142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 msgr2=0x7f22dc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 0x7f22dc198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 -- 172.21.15.37:0/4181654142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc19fe10 con 0x7f22dc102a00 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.433+0000 7f22e2297640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22d400eaa0 tx=0x7f22d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.434+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d400cdb0 con 0x7f22dc102a00 2024-04-25T13:23:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.434+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d4005020 con 0x7f22dc102a00 2024-04-25T13:23:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.434+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a0100 con 0x7f22dc102a00 2024-04-25T13:23:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.434+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40116d0 con 0x7f22dc102a00 2024-04-25T13:23:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.434+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a0650 con 0x7f22dc102a00 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.435+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc1a0910 con 0x7f22dc102a00 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.436+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f22d40051d0 con 0x7f22dc102a00 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.436+0000 7f22d2ffd640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 0x7f22c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.437+0000 7f22e1a96640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 0x7f22c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.437+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f22d40d0360 con 0x7f22dc102a00 2024-04-25T13:23:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.437+0000 7f22e1a96640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 0x7f22c80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009230 tx=0x7f22d80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:36.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.440+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22d4016030 con 0x7f22dc102a00 2024-04-25T13:23:36.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.668+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22dc1a0ae0 con 0x7f22dc102a00 2024-04-25T13:23:36.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.668+0000 7f22d2ffd640 1 -- 172.21.15.37:0/4181654142 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f22d409cc10 con 0x7f22dc102a00 2024-04-25T13:23:36.669 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:36.669 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:23:36.669 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 msgr2=0x7f22c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 0x7f22c80a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009230 tx=0x7f22d80314a0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 msgr2=0x7f22dc19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22d400eaa0 tx=0x7f22d400ef60 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 shutdown_connections 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f22c80a3c30 0x7f22c80a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22dc102a00 0x7f22dc19f7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f22dc102040 0x7f22dc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 --2- 172.21.15.37:0/4181654142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f22dc100e60 0x7f22dc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 >> 172.21.15.37:0/4181654142 conn(0x7f22dc0fc460 msgr2=0x7f22dc06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 shutdown_connections 2024-04-25T13:23:36.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:36.670+0000 7f22e3cfa640 1 -- 172.21.15.37:0/4181654142 wait complete. 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.69:6832/579371697,v1:172.21.15.69:6833/579371697] up:boot 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: fsmap 1 up:standby 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: Deploying daemon mds.k on smithi044 2024-04-25T13:23:36.831 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:36 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/4181654142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.69:6832/579371697,v1:172.21.15.69:6833/579371697] up:boot 2024-04-25T13:23:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: fsmap 1 up:standby 2024-04-25T13:23:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: Deploying daemon mds.k on smithi044 2024-04-25T13:23:37.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:36 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4181654142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.69:6832/579371697,v1:172.21.15.69:6833/579371697] up:boot 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: fsmap 1 up:standby 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:37.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: Deploying daemon mds.k on smithi044 2024-04-25T13:23:37.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:36 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/4181654142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:38.140 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 systemd[1]: Starting Ceph mds.k for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:38.491 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 podman[61962]: 2024-04-25 13:23:38.125511448 +0000 UTC m=+0.018553897 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:38.492 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 podman[61962]: 2024-04-25 13:23:38.237859194 +0000 UTC m=+0.130901628 container create c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:23:38.492 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 podman[61962]: 2024-04-25 13:23:38.465411033 +0000 UTC m=+0.358453468 container init c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:38.492 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 podman[61962]: 2024-04-25 13:23:38.468141941 +0000 UTC m=+0.361184375 container start c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:23:38.492 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mds[61982]: -- 172.21.15.44:0/2166226360 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3a951ba40 con 0x55f3a94ee800 2024-04-25T13:23:38.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:38.778 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k[61978]: starting mds.k at 2024-04-25T13:23:38.778 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 bash[61962]: c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 2024-04-25T13:23:38.778 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:23:38 smithi044 systemd[1]: Started Ceph mds.k for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- 172.21.15.37:0/427726715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68102a90 msgr2=0x7fac68104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 --2- 172.21.15.37:0/427726715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68102a90 0x7fac68104e80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009900 tx=0x7fac5c02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:38.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- 172.21.15.37:0/427726715 shutdown_connections 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 --2- 172.21.15.37:0/427726715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68107cf0 0x7fac6810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 --2- 172.21.15.37:0/427726715 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 0x7fac681077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 --2- 172.21.15.37:0/427726715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68102a90 0x7fac68104e80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- 172.21.15.37:0/427726715 >> 172.21.15.37:0/427726715 conn(0x7fac680fc420 msgr2=0x7fac680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- 172.21.15.37:0/427726715 shutdown_connections 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- 172.21.15.37:0/427726715 wait complete. 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 Processor -- start 2024-04-25T13:23:38.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.792+0000 7fac6eb22640 1 -- start start 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 0x7fac68102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 0x7fac68100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810db40 con 0x7fac68107cf0 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810d9c0 con 0x7fac68102a90 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac6810dcc0 con 0x7fac681053c0 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43054/0 (socket says 172.21.15.37:43054) 2024-04-25T13:23:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 -- 172.21.15.37:0/1547312743 learned_addr learned my addr 172.21.15.37:0/1547312743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6c8be640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 0x7fac68102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac67fff640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 0x7fac68100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 -- 172.21.15.37:0/1547312743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 msgr2=0x7fac68100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 0x7fac68100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 -- 172.21.15.37:0/1547312743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 msgr2=0x7fac68102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 0x7fac68102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 -- 172.21.15.37:0/1547312743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac68101810 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6c8be640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 0x7fac68102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6d0bf640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac6000eb10 tx=0x7fac6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60004b20 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac681a2310 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac60004cc0 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.793+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60010690 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.794+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681a2860 con 0x7fac68107cf0 2024-04-25T13:23:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.795+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac680fdf10 con 0x7fac68107cf0 2024-04-25T13:23:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.796+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac60005020 con 0x7fac68107cf0 2024-04-25T13:23:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.796+0000 7fac65ffb640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 0x7fac580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.796+0000 7fac6c8be640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 0x7fac580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.797+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fac600d0550 con 0x7fac68107cf0 2024-04-25T13:23:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.797+0000 7fac6c8be640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 0x7fac580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fac5c031500 tx=0x7fac5c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:38.801+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac6009d3d0 con 0x7fac68107cf0 2024-04-25T13:23:39.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.028+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac68062b50 con 0x7fac68107cf0 2024-04-25T13:23:39.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.029+0000 7fac65ffb640 1 -- 172.21.15.37:0/1547312743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7fac6009ccd0 con 0x7fac68107cf0 2024-04-25T13:23:39.029 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:39.030 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:23:39.030 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-25T13:23:39.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 msgr2=0x7fac580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:39.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 0x7fac580a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fac5c031500 tx=0x7fac5c005a20 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 msgr2=0x7fac68100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fac6000eb10 tx=0x7fac6000c2d0 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 shutdown_connections 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac580a3bb0 0x7fac580a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac68107cf0 0x7fac68100f80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac681053c0 0x7fac68100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 --2- 172.21.15.37:0/1547312743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac68102a90 0x7fac68102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 >> 172.21.15.37:0/1547312743 conn(0x7fac680fc420 msgr2=0x7fac68103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 shutdown_connections 2024-04-25T13:23:39.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:39.031+0000 7fac6eb22640 1 -- 172.21.15.37:0/1547312743 wait complete. 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:39.042 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:38 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:38 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:38 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:39.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:39 smithi044 ceph-mon[32590]: Deploying daemon mds.j on smithi037 2024-04-25T13:23:39.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:39 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6832/2542542028,v1:172.21.15.44:6833/2542542028] up:boot 2024-04-25T13:23:39.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:39 smithi044 ceph-mon[32590]: fsmap 2 up:standby 2024-04-25T13:23:39.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:39 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:23:39.875 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:39 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1547312743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:40.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:39 smithi037 ceph-mon[29204]: Deploying daemon mds.j on smithi037 2024-04-25T13:23:40.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:39 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6832/2542542028,v1:172.21.15.44:6833/2542542028] up:boot 2024-04-25T13:23:40.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:39 smithi037 ceph-mon[29204]: fsmap 2 up:standby 2024-04-25T13:23:40.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:39 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:23:40.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:39 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1547312743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:40.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:39 smithi069 ceph-mon[32534]: Deploying daemon mds.j on smithi037 2024-04-25T13:23:40.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:39 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6832/2542542028,v1:172.21.15.44:6833/2542542028] up:boot 2024-04-25T13:23:40.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:39 smithi069 ceph-mon[32534]: fsmap 2 up:standby 2024-04-25T13:23:40.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:39 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:23:40.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:39 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1547312743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:41.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:40 smithi037 ceph-mon[29204]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:41.026 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:40 smithi037 systemd[1]: Starting Ceph mds.j for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:41.044 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:40 smithi069 ceph-mon[32534]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:40 smithi044 ceph-mon[32590]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.150+0000 7f91ca005640 1 -- 172.21.15.37:0/4242591629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 msgr2=0x7f91c4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.150+0000 7f91ca005640 1 --2- 172.21.15.37:0/4242591629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c4102000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009ca0 tx=0x7f91bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- 172.21.15.37:0/4242591629 shutdown_connections 2024-04-25T13:23:41.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- 172.21.15.37:0/4242591629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c4102000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- 172.21.15.37:0/4242591629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 0x7f91c4101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- 172.21.15.37:0/4242591629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c4100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- 172.21.15.37:0/4242591629 >> 172.21.15.37:0/4242591629 conn(0x7f91c40fc420 msgr2=0x7f91c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- 172.21.15.37:0/4242591629 shutdown_connections 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- 172.21.15.37:0/4242591629 wait complete. 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 Processor -- start 2024-04-25T13:23:41.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- start start 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 0x7f91c419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.151+0000 7f91ca005640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f91c410fd40 con 0x7f91c4101b80 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91ca005640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91c410fbc0 con 0x7f91c4100880 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91ca005640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f91c410fec0 con 0x7f91c41011e0 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38340/0 (socket says 172.21.15.37:38340) 2024-04-25T13:23:41.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 -- 172.21.15.37:0/1466324861 learned_addr learned my addr 172.21.15.37:0/1466324861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c37fe640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c2ffd640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 0x7f91c419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 -- 172.21.15.37:0/1466324861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 msgr2=0x7f91c419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 0x7f91c419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 -- 172.21.15.37:0/1466324861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 msgr2=0x7f91c419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 -- 172.21.15.37:0/1466324861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c41a1fa0 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c37fe640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c3fff640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005e30 tx=0x7f91bc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc004240 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c41a2230 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91bc003e10 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc030e70 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.152+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c41a2800 con 0x7f91c4101b80 2024-04-25T13:23:41.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.153+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c410b740 con 0x7f91c4101b80 2024-04-25T13:23:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.155+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f91bc0305e0 con 0x7f91c4101b80 2024-04-25T13:23:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.155+0000 7f91c0ff9640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 0x7f91ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.155+0000 7f91c37fe640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 0x7f91ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.155+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f91bc0f1730 con 0x7f91c4101b80 2024-04-25T13:23:41.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.156+0000 7f91c37fe640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 0x7f91ac0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91b80058c0 tx=0x7f91b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.161+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91bc0098b0 con 0x7f91c4101b80 2024-04-25T13:23:41.285 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 podman[69741]: 2024-04-25 13:23:41.023440908 +0000 UTC m=+0.106487129 container create 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T13:23:41.286 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 podman[69741]: 2024-04-25 13:23:40.936212807 +0000 UTC m=+0.019258999 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:41.286 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 podman[69741]: 2024-04-25 13:23:41.259239987 +0000 UTC m=+0.342286167 container init 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T13:23:41.286 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 podman[69741]: 2024-04-25 13:23:41.262222052 +0000 UTC m=+0.345268232 container start 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T13:23:41.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.394+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91c402d5c0 con 0x7f91c4101b80 2024-04-25T13:23:41.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.395+0000 7f91c0ff9640 1 -- 172.21.15.37:0/1466324861 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f91c402d5c0 con 0x7f91c4101b80 2024-04-25T13:23:41.395 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:41.395 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:23:41.396 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 msgr2=0x7f91ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 0x7f91ac0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91b80058c0 tx=0x7f91b8005850 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 msgr2=0x7f91c41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005e30 tx=0x7f91bc003c20 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 shutdown_connections 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f91ac0a3bb0 0x7f91ac0a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91c4101b80 0x7f91c41a1930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f91c41011e0 0x7f91c419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 --2- 172.21.15.37:0/1466324861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91c4100880 0x7f91c419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 >> 172.21.15.37:0/1466324861 conn(0x7f91c40fc420 msgr2=0x7f91c40fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 shutdown_connections 2024-04-25T13:23:41.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:41.397+0000 7f91ca005640 1 -- 172.21.15.37:0/1466324861 wait complete. 2024-04-25T13:23:41.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mds[69799]: -- 172.21.15.37:0/186749789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f8f2a9a40 con 0x558f8f27cc00 2024-04-25T13:23:41.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j[69776]: starting mds.j at 2024-04-25T13:23:41.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 bash[69741]: 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a 2024-04-25T13:23:41.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:23:41 smithi037 systemd[1]: Started Ceph mds.j for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1466324861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:42.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:41 smithi037 ceph-mon[29204]: Deploying daemon mds.d on smithi037 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1466324861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:41 smithi069 ceph-mon[32534]: Deploying daemon mds.d on smithi037 2024-04-25T13:23:42.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1466324861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:42.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:41 smithi044 ceph-mon[32590]: Deploying daemon mds.d on smithi037 2024-04-25T13:23:43.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:42 smithi037 ceph-mon[29204]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:43.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:42 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.37:6834/1814974020,v1:172.21.15.37:6835/1814974020] up:boot 2024-04-25T13:23:43.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:42 smithi037 ceph-mon[29204]: fsmap 3 up:standby 2024-04-25T13:23:43.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:42 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:23:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:42 smithi069 ceph-mon[32534]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:42 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.37:6834/1814974020,v1:172.21.15.37:6835/1814974020] up:boot 2024-04-25T13:23:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:42 smithi069 ceph-mon[32534]: fsmap 3 up:standby 2024-04-25T13:23:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:42 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:23:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:42 smithi044 ceph-mon[32590]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:42 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.37:6834/1814974020,v1:172.21.15.37:6835/1814974020] up:boot 2024-04-25T13:23:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:42 smithi044 ceph-mon[32590]: fsmap 3 up:standby 2024-04-25T13:23:43.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:42 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:23:43.411 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 -- 172.21.15.37:0/1725548691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf741031b0 msgr2=0x7fbf74103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 --2- 172.21.15.37:0/1725548691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf741031b0 0x7fbf74103630 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009900 tx=0x7fbf6802a410 comp rx=0 tx=0).stop 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 -- 172.21.15.37:0/1725548691 shutdown_connections 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 --2- 172.21.15.37:0/1725548691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf74103b70 0x7fbf7410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 --2- 172.21.15.37:0/1725548691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf741031b0 0x7fbf74103630 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 --2- 172.21.15.37:0/1725548691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74101fc0 0x7fbf741023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 -- 172.21.15.37:0/1725548691 >> 172.21.15.37:0/1725548691 conn(0x7fbf740fd530 msgr2=0x7fbf740ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 -- 172.21.15.37:0/1725548691 shutdown_connections 2024-04-25T13:23:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.530+0000 7fbf79f44640 1 -- 172.21.15.37:0/1725548691 wait complete. 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 Processor -- start 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 -- start start 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf741031b0 0x7fbf7419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 0x7fbf741a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410db00 con 0x7fbf74101fc0 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410d980 con 0x7fbf741031b0 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf79f44640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7410dc80 con 0x7fbf74103b70 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38370/0 (socket says 172.21.15.37:38370) 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf737fe640 1 -- 172.21.15.37:0/3969078547 learned_addr learned my addr 172.21.15.37:0/3969078547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf72ffd640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf741031b0 0x7fbf7419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.531+0000 7fbf73fff640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 0x7fbf741a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 -- 172.21.15.37:0/3969078547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 msgr2=0x7fbf741a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 0x7fbf741a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 -- 172.21.15.37:0/3969078547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf741031b0 msgr2=0x7fbf7419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf741031b0 0x7fbf7419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 -- 172.21.15.37:0/3969078547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741a1fa0 con 0x7fbf74101fc0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf73fff640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 0x7fbf741a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf737fe640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400ebc0 tx=0x7fbf6400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64003b70 con 0x7fbf74101fc0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a2290 con 0x7fbf74101fc0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf64003d10 con 0x7fbf74101fc0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64019800 con 0x7fbf74101fc0 2024-04-25T13:23:43.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.532+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a2800 con 0x7fbf74101fc0 2024-04-25T13:23:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.533+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf741a2ac0 con 0x7fbf74101fc0 2024-04-25T13:23:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.534+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf640199a0 con 0x7fbf74101fc0 2024-04-25T13:23:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.535+0000 7fbf70ff9640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.535+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fbf640d0890 con 0x7fbf74101fc0 2024-04-25T13:23:43.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.535+0000 7fbf72ffd640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:43.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.536+0000 7fbf72ffd640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf680314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:43.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.540+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf6409d710 con 0x7fbf74101fc0 2024-04-25T13:23:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.774+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf741a2c90 con 0x7fbf74101fc0 2024-04-25T13:23:43.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.775+0000 7fbf70ff9640 1 -- 172.21.15.37:0/3969078547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7fbf6409d010 con 0x7fbf74101fc0 2024-04-25T13:23:43.776 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:43.777 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:23:43.777 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 msgr2=0x7fbf5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf68000c00 tx=0x7fbf680314a0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 msgr2=0x7fbf7419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400ebc0 tx=0x7fbf6400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 shutdown_connections 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf74103b70 0x7fbf741a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf741031b0 0x7fbf7419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbf5c0a3bb0 0x7fbf5c0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 --2- 172.21.15.37:0/3969078547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf74101fc0 0x7fbf7419a380 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 >> 172.21.15.37:0/3969078547 conn(0x7fbf740fd530 msgr2=0x7fbf74108950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 shutdown_connections 2024-04-25T13:23:43.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:43.777+0000 7fbf79f44640 1 -- 172.21.15.37:0/3969078547 wait complete. 2024-04-25T13:23:43.790 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:43 smithi037 podman[70127]: 2024-04-25 13:23:43.678057332 +0000 UTC m=+0.127042824 container create e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T13:23:44.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:43 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3969078547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:44.101 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:43 smithi037 podman[70127]: 2024-04-25 13:23:43.931062687 +0000 UTC m=+0.380048179 container init e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:23:44.102 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:43 smithi037 podman[70127]: 2024-04-25 13:23:43.934487825 +0000 UTC m=+0.383473317 container start e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T13:23:44.102 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:43 smithi037 ceph-mds[70149]: -- 172.21.15.37:0/252489304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a16af9a40 con 0x560a16acc800 2024-04-25T13:23:44.102 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:43 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d[70145]: starting mds.d at 2024-04-25T13:23:44.102 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:44 smithi037 bash[70127]: e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 2024-04-25T13:23:44.102 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:23:44 smithi037 systemd[1]: Started Ceph mds.d for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:44.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:43 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3969078547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:44.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:43 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3969078547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: Deploying daemon mds.e on smithi044 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.37:6836/3506634050,v1:172.21.15.37:6837/3506634050] up:boot 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: fsmap 4 up:standby 2024-04-25T13:23:45.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:45 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:45.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: Deploying daemon mds.e on smithi044 2024-04-25T13:23:45.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.37:6836/3506634050,v1:172.21.15.37:6837/3506634050] up:boot 2024-04-25T13:23:45.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: fsmap 4 up:standby 2024-04-25T13:23:45.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:45 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:23:45.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: Deploying daemon mds.e on smithi044 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.37:6836/3506634050,v1:172.21.15.37:6837/3506634050] up:boot 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: fsmap 4 up:standby 2024-04-25T13:23:45.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:45 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:23:45.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:45.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 -- 172.21.15.37:0/895711983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 msgr2=0x7fc3fc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 --2- 172.21.15.37:0/895711983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc1023d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009900 tx=0x7fc3ec02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 -- 172.21.15.37:0/895711983 shutdown_connections 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 --2- 172.21.15.37:0/895711983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc103b60 0x7fc3fc10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 --2- 172.21.15.37:0/895711983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 --2- 172.21.15.37:0/895711983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc1023d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 -- 172.21.15.37:0/895711983 >> 172.21.15.37:0/895711983 conn(0x7fc3fc0fd540 msgr2=0x7fc3fc0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 -- 172.21.15.37:0/895711983 shutdown_connections 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 -- 172.21.15.37:0/895711983 wait complete. 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.908+0000 7fc401b29640 1 Processor -- start 2024-04-25T13:23:45.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 -- start start 2024-04-25T13:23:45.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc103b60 0x7fc3fc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc10daf0 con 0x7fc3fc101fb0 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc10d970 con 0x7fc3fc103b60 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc401b29640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc10dc70 con 0x7fc3fc1031a0 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38408/0 (socket says 172.21.15.37:38408) 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 -- 172.21.15.37:0/471222062 learned_addr learned my addr 172.21.15.37:0/471222062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fbfff640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc103b60 0x7fc3fc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3faffd640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 -- 172.21.15.37:0/471222062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 msgr2=0x7fc3fc074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 -- 172.21.15.37:0/471222062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc103b60 msgr2=0x7fc3fc074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc103b60 0x7fc3fc074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:45.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 -- 172.21.15.37:0/471222062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc075110 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3faffd640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.909+0000 7fc3fb7fe640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec006010 tx=0x7fc3ec004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.910+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec03f030 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.910+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec003cb0 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.910+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc1a2230 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.910+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec035070 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.910+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc1a27d0 con 0x7fc3fc101fb0 2024-04-25T13:23:45.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.911+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc1092d0 con 0x7fc3fc101fb0 2024-04-25T13:23:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.912+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3ec042030 con 0x7fc3fc101fb0 2024-04-25T13:23:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.913+0000 7fc3f8ff9640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 0x7fc3e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.913+0000 7fc3faffd640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 0x7fc3e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.913+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fc3ec0f09d0 con 0x7fc3fc101fb0 2024-04-25T13:23:45.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.914+0000 7fc3faffd640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 0x7fc3e40a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc075a70 tx=0x7fc3f000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:45.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:45.917+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3ec0bd850 con 0x7fc3fc101fb0 2024-04-25T13:23:46.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.145+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3fc02d5c0 con 0x7fc3fc101fb0 2024-04-25T13:23:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.146+0000 7fc3f8ff9640 1 -- 172.21.15.37:0/471222062 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fc3ec003e60 con 0x7fc3fc101fb0 2024-04-25T13:23:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:46.147 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:23:46.147 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-04-25T13:23:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 msgr2=0x7fc3e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 0x7fc3e40a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc075a70 tx=0x7fc3f000a5c0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 msgr2=0x7fc3fc075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec006010 tx=0x7fc3ec004430 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 shutdown_connections 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3fc103b60 0x7fc3fc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3fc1031a0 0x7fc3fc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc3e40a3c30 0x7fc3e40a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 --2- 172.21.15.37:0/471222062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3fc101fb0 0x7fc3fc075d00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 >> 172.21.15.37:0/471222062 conn(0x7fc3fc0fd540 msgr2=0x7fc3fc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 shutdown_connections 2024-04-25T13:23:46.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:46.148+0000 7fc401b29640 1 -- 172.21.15.37:0/471222062 wait complete. 2024-04-25T13:23:46.390 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 systemd[1]: Starting Ceph mds.e for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:46.744 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 podman[62282]: 2024-04-25 13:23:46.349167014 +0000 UTC m=+0.018695802 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:46.744 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 podman[62282]: 2024-04-25 13:23:46.458787491 +0000 UTC m=+0.128316253 container create 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:23:46.744 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 podman[62282]: 2024-04-25 13:23:46.744523723 +0000 UTC m=+0.414052483 container init 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:23:47.067 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 podman[62282]: 2024-04-25 13:23:46.74748084 +0000 UTC m=+0.417009600 container start 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:47.067 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 ceph-mds[62301]: -- 172.21.15.44:0/2834907811 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55947e345a40 con 0x55947e318800 2024-04-25T13:23:47.067 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e[62297]: starting mds.e at 2024-04-25T13:23:47.068 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 bash[62282]: 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 2024-04-25T13:23:47.068 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:23:46 smithi044 systemd[1]: Started Ceph mds.e for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/471222062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:47.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:47.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:47 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/471222062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:47 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:47.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:47.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/471222062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:47.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:47.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:47 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:48.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 -- 172.21.15.37:0/662804662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 msgr2=0x7f9938102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 --2- 172.21.15.37:0/662804662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f9938102870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9934009900 tx=0x7f993402a410 comp rx=0 tx=0).stop 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 -- 172.21.15.37:0/662804662 shutdown_connections 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 --2- 172.21.15.37:0/662804662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f99381093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 --2- 172.21.15.37:0/662804662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 --2- 172.21.15.37:0/662804662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f9938102870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 -- 172.21.15.37:0/662804662 >> 172.21.15.37:0/662804662 conn(0x7f99380fbbb0 msgr2=0x7f99380fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 -- 172.21.15.37:0/662804662 shutdown_connections 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.275+0000 7f99402a7640 1 -- 172.21.15.37:0/662804662 wait complete. 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 Processor -- start 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 -- start start 2024-04-25T13:23:48.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f9938198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f993810db20 con 0x7f9938102450 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f993810d9a0 con 0x7f9938101b40 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f99402a7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f993810dca0 con 0x7f9938101180 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e043640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38438/0 (socket says 172.21.15.37:38438) 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 -- 172.21.15.37:0/1400400082 learned_addr learned my addr 172.21.15.37:0/1400400082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993d842640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f9938198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 -- 172.21.15.37:0/1400400082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 msgr2=0x7f9938198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 -- 172.21.15.37:0/1400400082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 msgr2=0x7f9938198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f9938198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e844640 1 -- 172.21.15.37:0/1400400082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993819fe60 con 0x7f9938102450 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993d842640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f9938198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.276+0000 7f993e043640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f993e844640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f993000eaa0 tx=0x7f993000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000cdb0 con 0x7f9938102450 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9930005020 con 0x7f9938102450 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300116d0 con 0x7f9938102450 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381a0150 con 0x7f9938102450 2024-04-25T13:23:48.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.277+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381a06a0 con 0x7f9938102450 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.278+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99381083e0 con 0x7f9938102450 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.279+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f99300051d0 con 0x7f9938102450 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.280+0000 7f992f7fe640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 0x7f99240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.280+0000 7f993e043640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 0x7f99240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.280+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f99300d00f0 con 0x7f9938102450 2024-04-25T13:23:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.280+0000 7f993e043640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 0x7f99240a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9934009230 tx=0x7f99340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:48.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.284+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9930016030 con 0x7f9938102450 2024-04-25T13:23:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:48 smithi037 ceph-mon[29204]: Deploying daemon mds.f on smithi069 2024-04-25T13:23:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:48 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6834/1266567676,v1:172.21.15.44:6835/1266567676] up:boot 2024-04-25T13:23:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:48 smithi037 ceph-mon[29204]: fsmap 5 up:standby 2024-04-25T13:23:48.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:48 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:48 smithi069 ceph-mon[32534]: Deploying daemon mds.f on smithi069 2024-04-25T13:23:48.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:48 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6834/1266567676,v1:172.21.15.44:6835/1266567676] up:boot 2024-04-25T13:23:48.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:48 smithi069 ceph-mon[32534]: fsmap 5 up:standby 2024-04-25T13:23:48.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:48 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:48 smithi044 ceph-mon[32590]: Deploying daemon mds.f on smithi069 2024-04-25T13:23:48.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:48 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6834/1266567676,v1:172.21.15.44:6835/1266567676] up:boot 2024-04-25T13:23:48.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:48 smithi044 ceph-mon[32590]: fsmap 5 up:standby 2024-04-25T13:23:48.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:48 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:23:48.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.511+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9938062b50 con 0x7f9938102450 2024-04-25T13:23:48.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.512+0000 7f992f7fe640 1 -- 172.21.15.37:0/1400400082 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f993009cf70 con 0x7f9938102450 2024-04-25T13:23:48.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:48.514 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T13:23:48.514 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-04-25T13:23:48.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 msgr2=0x7f99240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 0x7f99240a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9934009230 tx=0x7f99340314a0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 msgr2=0x7f993819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f993000eaa0 tx=0x7f993000ef60 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 shutdown_connections 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99240a3c30 0x7f99240a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9938102450 0x7f993819f7f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9938101b40 0x7f9938198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 --2- 172.21.15.37:0/1400400082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9938101180 0x7f9938198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 >> 172.21.15.37:0/1400400082 conn(0x7f99380fbbb0 msgr2=0x7f99381055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 shutdown_connections 2024-04-25T13:23:48.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:48.514+0000 7f99402a7640 1 -- 172.21.15.37:0/1400400082 wait complete. 2024-04-25T13:23:49.079 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 systemd[1]: Starting Ceph mds.f for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:49.334 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:49 smithi069 ceph-mon[32534]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:49.334 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:49 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1400400082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:49.335 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 podman[62495]: 2024-04-25 13:23:49.126921625 +0000 UTC m=+0.017873328 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:49.335 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 podman[62495]: 2024-04-25 13:23:49.228830734 +0000 UTC m=+0.119782425 container create b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:23:49.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:49 smithi037 ceph-mon[29204]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:49.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:49 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1400400082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:49.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:49 smithi044 ceph-mon[32590]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:49.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:49 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1400400082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:49.885 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 podman[62495]: 2024-04-25 13:23:49.472726515 +0000 UTC m=+0.363678205 container init b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:23:49.885 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 podman[62495]: 2024-04-25 13:23:49.475468593 +0000 UTC m=+0.366420284 container start b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:49.885 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 ceph-mds[62515]: -- 172.21.15.69:0/2715284978 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56039a357a40 con 0x56039a32a800 2024-04-25T13:23:49.885 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f[62511]: starting mds.f at 2024-04-25T13:23:49.886 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 bash[62495]: b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 2024-04-25T13:23:49.886 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:23:49 smithi069 systemd[1]: Started Ceph mds.f for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:50.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:50.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 -- 172.21.15.37:0/2878026396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0102ad0 msgr2=0x7fb3b0104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/2878026396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0102ad0 0x7fb3b0104ec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009900 tx=0x7fb3a002a410 comp rx=0 tx=0).stop 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 -- 172.21.15.37:0/2878026396 shutdown_connections 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/2878026396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0107d30 0x7fb3b010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/2878026396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0105400 0x7fb3b01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/2878026396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0102ad0 0x7fb3b0104ec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 -- 172.21.15.37:0/2878026396 >> 172.21.15.37:0/2878026396 conn(0x7fb3b00fc420 msgr2=0x7fb3b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 -- 172.21.15.37:0/2878026396 shutdown_connections 2024-04-25T13:23:50.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.644+0000 7fb3b4acb640 1 -- 172.21.15.37:0/2878026396 wait complete. 2024-04-25T13:23:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 Processor -- start 2024-04-25T13:23:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 -- start start 2024-04-25T13:23:50.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0102ad0 0x7fb3b01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 0x7fb3b019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010db60 con 0x7fb3b0105400 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010d9e0 con 0x7fb3b0107d30 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3b4acb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010dce0 con 0x7fb3b0102ad0 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3add9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3add9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59700/0 (socket says 172.21.15.37:59700) 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3add9b640 1 -- 172.21.15.37:0/4131182476 learned_addr learned my addr 172.21.15.37:0/4131182476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3ae59c640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0102ad0 0x7fb3b01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.645+0000 7fb3aed9d640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 0x7fb3b019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 -- 172.21.15.37:0/4131182476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0102ad0 msgr2=0x7fb3b01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0102ad0 0x7fb3b01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 -- 172.21.15.37:0/4131182476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 msgr2=0x7fb3b019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:50.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 0x7fb3b019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 -- 172.21.15.37:0/4131182476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b019fde0 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3aed9d640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 0x7fb3b019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3add9b640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e9c0 tx=0x7fb3a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a400cd80 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b01a00d0 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a40054d0 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4010430 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.646+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01a0620 con 0x7fb3b0105400 2024-04-25T13:23:50.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.647+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b00fdf10 con 0x7fb3b0105400 2024-04-25T13:23:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.648+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3a4004fb0 con 0x7fb3b0105400 2024-04-25T13:23:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.649+0000 7fb39f7fe640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 0x7fb3980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.649+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fb3a40d05f0 con 0x7fb3b0105400 2024-04-25T13:23:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.649+0000 7fb3ae59c640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 0x7fb3980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:50.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.650+0000 7fb3ae59c640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 0x7fb3980a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0031500 tx=0x7fb3a0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:50.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.653+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3a4053ff0 con 0x7fb3b0105400 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: Deploying daemon mds.c on smithi069 2024-04-25T13:23:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:50.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.69:6834/2452024894,v1:172.21.15.69:6835/2452024894] up:boot 2024-04-25T13:23:50.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: fsmap 6 up:standby 2024-04-25T13:23:50.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:50 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:50.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.876+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b002d810 con 0x7fb3b0105400 2024-04-25T13:23:50.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.877+0000 7fb39f7fe640 1 -- 172.21.15.37:0/4131182476 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7fb3a409d470 con 0x7fb3b0105400 2024-04-25T13:23:50.878 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:50.879 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T13:23:50.879 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 msgr2=0x7fb3980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 0x7fb3980a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0031500 tx=0x7fb3a0005a20 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 msgr2=0x7fb3b0198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400e9c0 tx=0x7fb3a400ee80 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 shutdown_connections 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3b0107d30 0x7fb3b019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb3980a3bb0 0x7fb3980a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b0105400 0x7fb3b0198700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 --2- 172.21.15.37:0/4131182476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb3b0102ad0 0x7fb3b01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 >> 172.21.15.37:0/4131182476 conn(0x7fb3b00fc420 msgr2=0x7fb3b0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:50.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 shutdown_connections 2024-04-25T13:23:50.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:50.879+0000 7fb3b4acb640 1 -- 172.21.15.37:0/4131182476 wait complete. 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:50.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: Deploying daemon mds.c on smithi069 2024-04-25T13:23:50.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:50.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.69:6834/2452024894,v1:172.21.15.69:6835/2452024894] up:boot 2024-04-25T13:23:50.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: fsmap 6 up:standby 2024-04-25T13:23:50.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:50 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:50.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: Deploying daemon mds.c on smithi069 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.69:6834/2452024894,v1:172.21.15.69:6835/2452024894] up:boot 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: fsmap 6 up:standby 2024-04-25T13:23:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:50 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:23:51.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:51 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/4131182476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:51.886 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:51 smithi069 systemd[1]: Starting Ceph mds.c for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:52.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:51 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4131182476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:52.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:51 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/4131182476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:52.275 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 podman[62800]: 2024-04-25 13:23:51.909251361 +0000 UTC m=+0.017836668 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:52.275 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 podman[62800]: 2024-04-25 13:23:52.039675784 +0000 UTC m=+0.148261063 container create 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:52.275 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 podman[62800]: 2024-04-25 13:23:52.275585541 +0000 UTC m=+0.384170821 container init 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:52.635 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 podman[62800]: 2024-04-25 13:23:52.278249625 +0000 UTC m=+0.386834914 container start 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:23:52.635 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 ceph-mds[62819]: -- 172.21.15.69:0/3007818275 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d53855da40 con 0x55d538531000 2024-04-25T13:23:52.635 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c[62815]: starting mds.c at 2024-04-25T13:23:52.635 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 bash[62800]: 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 2024-04-25T13:23:52.635 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:23:52 smithi069 systemd[1]: Started Ceph mds.c for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:52.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 -- 172.21.15.37:0/2032873832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 msgr2=0x7f8948109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 --2- 172.21.15.37:0/2032873832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948109320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8940009ca0 tx=0x7f894002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 -- 172.21.15.37:0/2032873832 shutdown_connections 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 --2- 172.21.15.37:0/2032873832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948109320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 --2- 172.21.15.37:0/2032873832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948102040 0x7f89481024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 --2- 172.21.15.37:0/2032873832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948100e60 0x7f8948101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 -- 172.21.15.37:0/2032873832 >> 172.21.15.37:0/2032873832 conn(0x7f89480fc420 msgr2=0x7f89480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 -- 172.21.15.37:0/2032873832 shutdown_connections 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.994+0000 7f894dca9640 1 -- 172.21.15.37:0/2032873832 wait complete. 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 Processor -- start 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 -- start start 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948100e60 0x7f8948104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948102040 0x7f8948103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f894810ca10 con 0x7f8948102a00 2024-04-25T13:23:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f894810c890 con 0x7f8948100e60 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f894dca9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f894810cb90 con 0x7f8948102040 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59716/0 (socket says 172.21.15.37:59716) 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 -- 172.21.15.37:0/1070635081 learned_addr learned my addr 172.21.15.37:0/1070635081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f89477fe640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948100e60 0x7f8948104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8946ffd640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948102040 0x7f8948103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 -- 172.21.15.37:0/1070635081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948102040 msgr2=0x7f8948103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948102040 0x7f8948103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 -- 172.21.15.37:0/1070635081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948100e60 msgr2=0x7f8948104ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948100e60 0x7f8948104ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.995+0000 7f8947fff640 1 -- 172.21.15.37:0/1070635081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89481042f0 con 0x7f8948102a00 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f8947fff640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89400097b0 tx=0x7f8940031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940003bf0 con 0x7f8948102a00 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8940003d90 con 0x7f8948102a00 2024-04-25T13:23:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89481a2280 con 0x7f8948102a00 2024-04-25T13:23:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894002fc70 con 0x7f8948102a00 2024-04-25T13:23:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.996+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481a2820 con 0x7f8948102a00 2024-04-25T13:23:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.997+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89481a2b40 con 0x7f8948102a00 2024-04-25T13:23:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.998+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8940004040 con 0x7f8948102a00 2024-04-25T13:23:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.999+0000 7f8944ff9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 0x7f89300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.999+0000 7f89477fe640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 0x7f89300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.999+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f89400f1450 con 0x7f8948102a00 2024-04-25T13:23:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:52.999+0000 7f89477fe640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 0x7f89300a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8948101bb0 tx=0x7f8938005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:53.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.004+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89400be2d0 con 0x7f8948102a00 2024-04-25T13:23:53.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.247+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89481a3030 con 0x7f8948102a00 2024-04-25T13:23:53.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.248+0000 7f8944ff9640 1 -- 172.21.15.37:0/1070635081 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f89400bdbd0 con 0x7f8948102a00 2024-04-25T13:23:53.249 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:53.249 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:23:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-04-25T13:23:53.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 msgr2=0x7f89300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 0x7f89300a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8948101bb0 tx=0x7f8938005f90 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 msgr2=0x7f8948103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89400097b0 tx=0x7f8940031840 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 shutdown_connections 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f89300a3c30 0x7f89300a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8948102a00 0x7f8948103a60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8948102040 0x7f8948103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 --2- 172.21.15.37:0/1070635081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8948100e60 0x7f8948104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 >> 172.21.15.37:0/1070635081 conn(0x7f89480fc420 msgr2=0x7f8948069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 shutdown_connections 2024-04-25T13:23:53.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:53.250+0000 7f894dca9640 1 -- 172.21.15.37:0/1070635081 wait complete. 2024-04-25T13:23:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: Deploying daemon mds.h on smithi044 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.69:6836/2736481323,v1:172.21.15.69:6837/2736481323] up:boot 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: fsmap 7 up:standby 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:53.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:53 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1070635081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:53.689 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:53.689 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: Deploying daemon mds.h on smithi044 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.69:6836/2736481323,v1:172.21.15.69:6837/2736481323] up:boot 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: fsmap 7 up:standby 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:53.690 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:53 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1070635081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:53.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:53.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: Deploying daemon mds.h on smithi044 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.69:6836/2736481323,v1:172.21.15.69:6837/2736481323] up:boot 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: fsmap 7 up:standby 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:23:53.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:53 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1070635081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:54.890 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:54 smithi044 systemd[1]: Starting Ceph mds.h for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:55.231 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:54 smithi044 podman[62600]: 2024-04-25 13:23:54.865164019 +0000 UTC m=+0.018459539 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:55.232 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:54 smithi044 podman[62600]: 2024-04-25 13:23:54.987781258 +0000 UTC m=+0.141076762 container create 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T13:23:55.232 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 podman[62600]: 2024-04-25 13:23:55.231931423 +0000 UTC m=+0.385226927 container init 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:23:55.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:55.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/1371008698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 msgr2=0x7f1c78107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/1371008698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78107750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009900 tx=0x7f1c6c02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/1371008698 shutdown_connections 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/1371008698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/1371008698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78107750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/1371008698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 0x7f1c78104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/1371008698 >> 172.21.15.37:0/1371008698 conn(0x7f1c780fc460 msgr2=0x7f1c780fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/1371008698 shutdown_connections 2024-04-25T13:23:55.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.376+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/1371008698 wait complete. 2024-04-25T13:23:55.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.377+0000 7f1c7dd5d640 1 Processor -- start 2024-04-25T13:23:55.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.377+0000 7f1c7dd5d640 1 -- start start 2024-04-25T13:23:55.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.377+0000 7f1c7dd5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 0x7f1c78198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:55.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.377+0000 7f1c7dd5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:55.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.377+0000 7f1c7dd5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c7dd5d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810db40 con 0x7f1c78105360 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c7dd5d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810d9c0 con 0x7f1c78107c90 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c7dd5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7810dcc0 con 0x7f1c78102a30 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 -- 172.21.15.37:0/3163828312 learned_addr learned my addr 172.21.15.37:0/3163828312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c77fff640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c777fe640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 0x7f1c78198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 -- 172.21.15.37:0/3163828312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 msgr2=0x7f1c78198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 0x7f1c78198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 -- 172.21.15.37:0/3163828312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 msgr2=0x7f1c7819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 -- 172.21.15.37:0/3163828312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7819fe40 con 0x7f1c78105360 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c77fff640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c76ffd640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c031500 tx=0x7f1c6c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c03f070 con 0x7f1c78105360 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c6c031c50 con 0x7f1c78105360 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781a00d0 con 0x7f1c78105360 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.378+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c035030 con 0x7f1c78105360 2024-04-25T13:23:55.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.379+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a05c0 con 0x7f1c78105360 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.380+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c780fdf50 con 0x7f1c78105360 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.381+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c6c042070 con 0x7f1c78105360 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.381+0000 7f1c74ff9640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 0x7f1c600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.382+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f1c6c0f0770 con 0x7f1c78105360 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.382+0000 7f1c777fe640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 0x7f1c600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:55.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.382+0000 7f1c777fe640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 0x7f1c600a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c68005fd0 tx=0x7f1c68005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:55.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.386+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c6c0bd5f0 con 0x7f1c78105360 2024-04-25T13:23:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:55 smithi037 ceph-mon[29204]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:55 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:55 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:55 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.614+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c7802d770 con 0x7f1c78105360 2024-04-25T13:23:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.615+0000 7f1c74ff9640 1 -- 172.21.15.37:0/3163828312 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f1c6c0bcef0 con 0x7f1c78105360 2024-04-25T13:23:55.616 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:55.616 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:23:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T13:23:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.616+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 msgr2=0x7f1c600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.616+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 0x7f1c600a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c68005fd0 tx=0x7f1c68005850 comp rx=0 tx=0).stop 2024-04-25T13:23:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.616+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 msgr2=0x7f1c78198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.616+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c031500 tx=0x7f1c6c003c10 comp rx=0 tx=0).stop 2024-04-25T13:23:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 shutdown_connections 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c78107c90 0x7f1c7819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1c600a3bb0 0x7f1c600a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c78105360 0x7f1c78198760 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 --2- 172.21.15.37:0/3163828312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c78102a30 0x7f1c78198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 >> 172.21.15.37:0/3163828312 conn(0x7f1c780fc460 msgr2=0x7f1c781034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 shutdown_connections 2024-04-25T13:23:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:55.617+0000 7f1c7dd5d640 1 -- 172.21.15.37:0/3163828312 wait complete. 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-mon[32590]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 podman[62600]: 2024-04-25 13:23:55.234733678 +0000 UTC m=+0.388029196 container start 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-mds[62619]: -- 172.21.15.44:0/3259830009 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55787bf0fa40 con 0x55787bee3000 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h[62615]: starting mds.h at 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 bash[62600]: 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 2024-04-25T13:23:55.640 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:23:55 smithi044 systemd[1]: Started Ceph mds.h for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:55.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:55 smithi069 ceph-mon[32534]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:55.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:55 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:55 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:55.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:55 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: Deploying daemon mds.i on smithi069 2024-04-25T13:23:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6836/690443752,v1:172.21.15.44:6837/690443752] up:boot 2024-04-25T13:23:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: fsmap 8 up:standby 2024-04-25T13:23:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:56.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:56 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3163828312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: Deploying daemon mds.i on smithi069 2024-04-25T13:23:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6836/690443752,v1:172.21.15.44:6837/690443752] up:boot 2024-04-25T13:23:56.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: fsmap 8 up:standby 2024-04-25T13:23:56.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:56.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:56 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3163828312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: Deploying daemon mds.i on smithi069 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6836/690443752,v1:172.21.15.44:6837/690443752] up:boot 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: fsmap 8 up:standby 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:23:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:56 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3163828312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:57.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:23:57.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:57 smithi069 ceph-mon[32534]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:57.635 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 systemd[1]: Starting Ceph mds.i for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 -- 172.21.15.37:0/3064460327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8481053c0 msgr2=0x7fb8481077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3064460327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8481053c0 0x7fb8481077b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009900 tx=0x7fb83c02a410 comp rx=0 tx=0).stop 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 -- 172.21.15.37:0/3064460327 shutdown_connections 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3064460327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848107cf0 0x7fb84810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3064460327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8481053c0 0x7fb8481077b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3064460327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb848102a90 0x7fb848104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 -- 172.21.15.37:0/3064460327 >> 172.21.15.37:0/3064460327 conn(0x7fb8480fc460 msgr2=0x7fb8480fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 -- 172.21.15.37:0/3064460327 shutdown_connections 2024-04-25T13:23:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.743+0000 7fb84cf19640 1 -- 172.21.15.37:0/3064460327 wait complete. 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 Processor -- start 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 -- start start 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 0x7fb848198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8481053c0 0x7fb848198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810dab0 con 0x7fb848107cf0 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810d930 con 0x7fb8481053c0 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb84cf19640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810dc30 con 0x7fb848102a90 2024-04-25T13:23:57.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.744+0000 7fb846d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59774/0 (socket says 172.21.15.37:59774) 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 -- 172.21.15.37:0/3108471053 learned_addr learned my addr 172.21.15.37:0/3108471053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb845d9b640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8481053c0 0x7fb848198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb84659c640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 0x7fb848198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 -- 172.21.15.37:0/3108471053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 msgr2=0x7fb848198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 0x7fb848198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 -- 172.21.15.37:0/3108471053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8481053c0 msgr2=0x7fb848198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8481053c0 0x7fb848198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 -- 172.21.15.37:0/3108471053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84819fd20 con 0x7fb848107cf0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb84659c640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 0x7fb848198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb846d9d640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb84000cce0 tx=0x7fb84000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84000f450 con 0x7fb848107cf0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84819ffb0 con 0x7fb848107cf0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb84000faa0 con 0x7fb848107cf0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840018b70 con 0x7fb848107cf0 2024-04-25T13:23:57.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.745+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a0480 con 0x7fb848107cf0 2024-04-25T13:23:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.746+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8480fdf50 con 0x7fb848107cf0 2024-04-25T13:23:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.747+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb840018d10 con 0x7fb848107cf0 2024-04-25T13:23:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.748+0000 7fb8377fe640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 0x7fb8300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:23:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.748+0000 7fb84659c640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 0x7fb8300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:23:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.748+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fb8400d04c0 con 0x7fb848107cf0 2024-04-25T13:23:57.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.748+0000 7fb84659c640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 0x7fb8300a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb83800abe0 tx=0x7fb83800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:23:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.752+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb840014030 con 0x7fb848107cf0 2024-04-25T13:23:57.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:57 smithi037 ceph-mon[29204]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:57.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:57 smithi044 ceph-mon[32590]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:57.953 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 podman[63107]: 2024-04-25 13:23:57.603318293 +0000 UTC m=+0.018085596 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:23:57.953 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 podman[63107]: 2024-04-25 13:23:57.726261478 +0000 UTC m=+0.141028796 container create 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T13:23:57.953 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 podman[63107]: 2024-04-25 13:23:57.9535679 +0000 UTC m=+0.368335176 container init 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:23:57.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.979+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb84802d5c0 con 0x7fb848107cf0 2024-04-25T13:23:57.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.980+0000 7fb8377fe640 1 -- 172.21.15.37:0/3108471053 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7fb84009d330 con 0x7fb848107cf0 2024-04-25T13:23:57.981 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:23:57.981 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T13:23:57.982 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 msgr2=0x7fb8300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 0x7fb8300a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb83800abe0 tx=0x7fb83800a5c0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 msgr2=0x7fb84819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb84000cce0 tx=0x7fb84000a620 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 shutdown_connections 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8300a3bb0 0x7fb8300a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb848107cf0 0x7fb84819f6b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8481053c0 0x7fb848198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 --2- 172.21.15.37:0/3108471053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb848102a90 0x7fb848198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 >> 172.21.15.37:0/3108471053 conn(0x7fb8480fc460 msgr2=0x7fb8481033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 shutdown_connections 2024-04-25T13:23:57.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:23:57.983+0000 7fb84cf19640 1 -- 172.21.15.37:0/3108471053 wait complete. 2024-04-25T13:23:58.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 podman[63107]: 2024-04-25 13:23:57.956317132 +0000 UTC m=+0.371084411 container start 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:23:58.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 ceph-mds[63127]: -- 172.21.15.69:0/583248691 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594dfc65a40 con 0x5594dfc39000 2024-04-25T13:23:58.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: starting mds.i at 2024-04-25T13:23:58.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:58 smithi069 bash[63107]: 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 2024-04-25T13:23:58.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:23:58 smithi069 systemd[1]: Started Ceph mds.i for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3108471053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:58 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3108471053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:58 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:58.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3108471053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:23:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:58 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:23:59.778 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:59 smithi037 ceph-mon[29204]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:59.778 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:59 smithi037 ceph-mon[29204]: Deploying daemon mds.g on smithi037 2024-04-25T13:23:59.778 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:59 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.69:6838/1356630935,v1:172.21.15.69:6839/1356630935] up:boot 2024-04-25T13:23:59.778 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:59 smithi037 ceph-mon[29204]: fsmap 9 up:standby 2024-04-25T13:23:59.778 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:23:59 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:59 smithi069 ceph-mon[32534]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:59 smithi069 ceph-mon[32534]: Deploying daemon mds.g on smithi037 2024-04-25T13:23:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:59 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.69:6838/1356630935,v1:172.21.15.69:6839/1356630935] up:boot 2024-04-25T13:23:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:59 smithi069 ceph-mon[32534]: fsmap 9 up:standby 2024-04-25T13:23:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:23:59 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:59 smithi044 ceph-mon[32590]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:23:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:59 smithi044 ceph-mon[32590]: Deploying daemon mds.g on smithi037 2024-04-25T13:23:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:59 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.69:6838/1356630935,v1:172.21.15.69:6839/1356630935] up:boot 2024-04-25T13:23:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:59 smithi044 ceph-mon[32590]: fsmap 9 up:standby 2024-04-25T13:23:59.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:23:59 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:23:59.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:24:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- 172.21.15.37:0/2058246119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c107d30 msgr2=0x7fa27c10a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 --2- 172.21.15.37:0/2058246119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c107d30 0x7fa27c10a120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa274009ca0 tx=0x7fa27402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- 172.21.15.37:0/2058246119 shutdown_connections 2024-04-25T13:24:00.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 --2- 172.21.15.37:0/2058246119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c107d30 0x7fa27c10a120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 --2- 172.21.15.37:0/2058246119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c105400 0x7fa27c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 --2- 172.21.15.37:0/2058246119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c102ad0 0x7fa27c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- 172.21.15.37:0/2058246119 >> 172.21.15.37:0/2058246119 conn(0x7fa27c0fc420 msgr2=0x7fa27c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- 172.21.15.37:0/2058246119 shutdown_connections 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- 172.21.15.37:0/2058246119 wait complete. 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 Processor -- start 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.261+0000 7fa284693640 1 -- start start 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c105400 0x7fa27c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 0x7fa27c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10db50 con 0x7fa27c102ad0 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10d9d0 con 0x7fa27c105400 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10dcd0 con 0x7fa27c107d30 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:00.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41760/0 (socket says 172.21.15.37:41760) 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 -- 172.21.15.37:0/3301424275 learned_addr learned my addr 172.21.15.37:0/3301424275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa281c2e640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c105400 0x7fa27c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa282c30640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 0x7fa27c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 -- 172.21.15.37:0/3301424275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 msgr2=0x7fa27c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 0x7fa27c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 -- 172.21.15.37:0/3301424275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c105400 msgr2=0x7fa27c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c105400 0x7fa27c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 -- 172.21.15.37:0/3301424275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c19fdf0 con 0x7fa27c102ad0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa282c30640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 0x7fa27c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa28242f640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa27800d950 tx=0x7fa27800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278017070 con 0x7fa27c102ad0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa278009c30 con 0x7fa27c102ad0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27800f620 con 0x7fa27c102ad0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c1a00e0 con 0x7fa27c102ad0 2024-04-25T13:24:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.262+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c1a0660 con 0x7fa27c102ad0 2024-04-25T13:24:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.263+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c0fdf10 con 0x7fa27c102ad0 2024-04-25T13:24:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.265+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa278010460 con 0x7fa27c102ad0 2024-04-25T13:24:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.265+0000 7fa2737fe640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.265+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fa2780cfc40 con 0x7fa27c102ad0 2024-04-25T13:24:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.265+0000 7fa281c2e640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.266+0000 7fa281c2e640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 0x7fa2680a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa27c1996e0 tx=0x7fa26c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:00.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.270+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa27809cac0 con 0x7fa27c102ad0 2024-04-25T13:24:00.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.506+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa27c02d5c0 con 0x7fa27c102ad0 2024-04-25T13:24:00.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.507+0000 7fa2737fe640 1 -- 172.21.15.37:0/3301424275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7989 (secure 0 0 0) 0x7fa278009de0 con 0x7fa27c102ad0 2024-04-25T13:24:00.507 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:00.507 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:24:00.507 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-04-25T13:24:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 msgr2=0x7fa2680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 0x7fa2680a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa27c1996e0 tx=0x7fa26c008040 comp rx=0 tx=0).stop 2024-04-25T13:24:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 msgr2=0x7fa27c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa27800d950 tx=0x7fa27800de10 comp rx=0 tx=0).stop 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 shutdown_connections 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa27c107d30 0x7fa27c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa27c105400 0x7fa27c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa2680a3bb0 0x7fa2680a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 --2- 172.21.15.37:0/3301424275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa27c102ad0 0x7fa27c1981d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 >> 172.21.15.37:0/3301424275 conn(0x7fa27c0fc420 msgr2=0x7fa27c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 shutdown_connections 2024-04-25T13:24:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:00.509+0000 7fa284693640 1 -- 172.21.15.37:0/3301424275 wait complete. 2024-04-25T13:24:00.521 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 systemd[1]: Starting Ceph mds.g for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:24:00.781 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 podman[70731]: 2024-04-25 13:24:00.543034292 +0000 UTC m=+0.018737391 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:00.781 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 podman[70731]: 2024-04-25 13:24:00.651869595 +0000 UTC m=+0.127572669 container create 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:24:01.101 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 podman[70731]: 2024-04-25 13:24:00.887669017 +0000 UTC m=+0.363372102 container init 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:24:01.101 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 podman[70731]: 2024-04-25 13:24:00.890581275 +0000 UTC m=+0.366284356 container start 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:24:01.101 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 ceph-mds[70750]: -- 172.21.15.37:0/3999078900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d9b153a40 con 0x564d9b126800 2024-04-25T13:24:01.101 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g[70746]: starting mds.g at 2024-04-25T13:24:01.102 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 bash[70731]: 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e 2024-04-25T13:24:01.102 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:24:00 smithi037 systemd[1]: Started Ceph mds.g for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3301424275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:01.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:01 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3301424275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:01 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3301424275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:01.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:01 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:02.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:24:02.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- 172.21.15.37:0/767913910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 msgr2=0x7f85ac10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 --2- 172.21.15.37:0/767913910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac10a0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009ca0 tx=0x7f85a402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- 172.21.15.37:0/767913910 shutdown_connections 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 --2- 172.21.15.37:0/767913910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac10a0e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 --2- 172.21.15.37:0/767913910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 0x7f85ac107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 --2- 172.21.15.37:0/767913910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 0x7f85ac104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- 172.21.15.37:0/767913910 >> 172.21.15.37:0/767913910 conn(0x7f85ac0fc420 msgr2=0x7f85ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- 172.21.15.37:0/767913910 shutdown_connections 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- 172.21.15.37:0/767913910 wait complete. 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 Processor -- start 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.638+0000 7f85b437d640 1 -- start start 2024-04-25T13:24:02.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 0x7f85ac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 0x7f85ac1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac10db40 con 0x7f85ac107c90 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac10d9c0 con 0x7f85ac102a30 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac10dcc0 con 0x7f85ac105360 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41782/0 (socket says 172.21.15.37:41782) 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 -- 172.21.15.37:0/1348603471 learned_addr learned my addr 172.21.15.37:0/1348603471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b1918640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 0x7f85ac1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b2119640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 0x7f85ac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 -- 172.21.15.37:0/1348603471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 msgr2=0x7f85ac1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 0x7f85ac1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 -- 172.21.15.37:0/1348603471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 msgr2=0x7f85ac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 0x7f85ac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 -- 172.21.15.37:0/1348603471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac19fcd0 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b291a640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85a4033040 tx=0x7f85a4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004040 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4004690 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a402fd60 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac19ff60 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.639+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1a0470 con 0x7f85ac107c90 2024-04-25T13:24:02.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.640+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac0fdf10 con 0x7f85ac107c90 2024-04-25T13:24:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.642+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f85a40041e0 con 0x7f85ac107c90 2024-04-25T13:24:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.643+0000 7f85a37fe640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 0x7f85980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.643+0000 7f85b2119640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 0x7f85980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.643+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f85a40f1880 con 0x7f85ac107c90 2024-04-25T13:24:02.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.644+0000 7f85b2119640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 0x7f85980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85a80058c0 tx=0x7f85a800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:02.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.647+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85a40098b0 con 0x7f85ac107c90 2024-04-25T13:24:02.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:02 smithi037 ceph-mon[29204]: Deploying daemon mds.a on smithi037 2024-04-25T13:24:02.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:02 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.37:6838/3019792631,v1:172.21.15.37:6839/3019792631] up:boot 2024-04-25T13:24:02.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:02 smithi037 ceph-mon[29204]: fsmap 10 up:standby 2024-04-25T13:24:02.670 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:02 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:24:02.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.880+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85ac02d5c0 con 0x7f85ac107c90 2024-04-25T13:24:02.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.881+0000 7f85a37fe640 1 -- 172.21.15.37:0/1348603471 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8815 (secure 0 0 0) 0x7f85a40be660 con 0x7f85ac107c90 2024-04-25T13:24:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:02.882 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:24:02.882 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 msgr2=0x7f85980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 0x7f85980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85a80058c0 tx=0x7f85a800a4c0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 msgr2=0x7f85ac19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f85a4033040 tx=0x7f85a4003ba0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 shutdown_connections 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f85980a3c30 0x7f85980a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85ac107c90 0x7f85ac19f660 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85ac105360 0x7f85ac1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 --2- 172.21.15.37:0/1348603471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85ac102a30 0x7f85ac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 >> 172.21.15.37:0/1348603471 conn(0x7f85ac0fc420 msgr2=0x7f85ac103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:02.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 shutdown_connections 2024-04-25T13:24:02.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:02.883+0000 7f85b437d640 1 -- 172.21.15.37:0/1348603471 wait complete. 2024-04-25T13:24:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:02 smithi069 ceph-mon[32534]: Deploying daemon mds.a on smithi037 2024-04-25T13:24:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:02 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.37:6838/3019792631,v1:172.21.15.37:6839/3019792631] up:boot 2024-04-25T13:24:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:02 smithi069 ceph-mon[32534]: fsmap 10 up:standby 2024-04-25T13:24:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:02 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:24:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:02 smithi044 ceph-mon[32590]: Deploying daemon mds.a on smithi037 2024-04-25T13:24:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:02 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.37:6838/3019792631,v1:172.21.15.37:6839/3019792631] up:boot 2024-04-25T13:24:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:02 smithi044 ceph-mon[32590]: fsmap 10 up:standby 2024-04-25T13:24:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:02 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:24:03.342 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 systemd[1]: Starting Ceph mds.a for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:24:03.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:03 smithi037 ceph-mon[29204]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:03.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:03 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1348603471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:03.601 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 podman[71080]: 2024-04-25 13:24:03.245881253 +0000 UTC m=+0.018525845 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:24:03.601 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 podman[71080]: 2024-04-25 13:24:03.380544923 +0000 UTC m=+0.153189503 container create 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:24:03.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:03 smithi069 ceph-mon[32534]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:03.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:03 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1348603471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:03.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:03 smithi044 ceph-mon[32590]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:03.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:03 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1348603471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:04.101 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 podman[71080]: 2024-04-25 13:24:03.633163423 +0000 UTC m=+0.405808006 container init 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:24:04.101 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 podman[71080]: 2024-04-25 13:24:03.635894196 +0000 UTC m=+0.408538777 container start 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T13:24:04.101 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 ceph-mds[71099]: -- 172.21.15.37:0/1208124249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650c0cfba40 con 0x5650c0cce800 2024-04-25T13:24:04.101 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a[71095]: starting mds.a at 2024-04-25T13:24:04.101 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 bash[71080]: 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 2024-04-25T13:24:04.102 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:24:03 smithi037 systemd[1]: Started Ceph mds.a for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:24:04.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:24:05.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- 172.21.15.37:0/2721640396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c102a30 msgr2=0x7f6a3c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:05.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 --2- 172.21.15.37:0/2721640396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c102a30 0x7f6a3c104e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009900 tx=0x7f6a3002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:05.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- 172.21.15.37:0/2721640396 shutdown_connections 2024-04-25T13:24:05.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 --2- 172.21.15.37:0/2721640396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c107c90 0x7f6a3c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 --2- 172.21.15.37:0/2721640396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c105360 0x7f6a3c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 --2- 172.21.15.37:0/2721640396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c102a30 0x7f6a3c104e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- 172.21.15.37:0/2721640396 >> 172.21.15.37:0/2721640396 conn(0x7f6a3c0fc420 msgr2=0x7f6a3c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- 172.21.15.37:0/2721640396 shutdown_connections 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- 172.21.15.37:0/2721640396 wait complete. 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 Processor -- start 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.014+0000 7f6a42423640 1 -- start start 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c102a30 0x7f6a3c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c107c90 0x7f6a3c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10daf0 con 0x7f6a3c105360 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10d970 con 0x7f6a3c107c90 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a42423640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c10dc70 con 0x7f6a3c102a30 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c102a30 0x7f6a3c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:05.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41802/0 (socket says 172.21.15.37:41802) 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3785035709 learned_addr learned my addr 172.21.15.37:0/3785035709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a409c0640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c107c90 0x7f6a3c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3785035709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c102a30 msgr2=0x7f6a3c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c102a30 0x7f6a3c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3785035709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c107c90 msgr2=0x7f6a3c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c107c90 0x7f6a3c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.015+0000 7f6a3b7fe640 1 -- 172.21.15.37:0/3785035709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c19fdf0 con 0x7f6a3c105360 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a3b7fe640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d770 tx=0x7f6a2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c017070 con 0x7f6a3c105360 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c009c30 con 0x7f6a3c105360 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c1a00e0 con 0x7f6a3c105360 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f620 con 0x7f6a3c105360 2024-04-25T13:24:05.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.016+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1a0680 con 0x7f6a3c105360 2024-04-25T13:24:05.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.017+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3c0fdf10 con 0x7f6a3c105360 2024-04-25T13:24:05.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.018+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6a2c01a070 con 0x7f6a3c105360 2024-04-25T13:24:05.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.019+0000 7f6a397fa640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 0x7f6a240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:05.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.019+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 0x7f6a240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:05.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.019+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f6a2c0cfa30 con 0x7f6a3c105360 2024-04-25T13:24:05.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.019+0000 7f6a3bfff640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 0x7f6a240a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a300314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:05.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.023+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a2c009de0 con 0x7f6a3c105360 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: Deploying daemon mds.b on smithi044 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.37:6840/1127100718,v1:172.21.15.37:6841/1127100718] up:boot 2024-04-25T13:24:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: fsmap 11 up:standby 2024-04-25T13:24:05.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:04 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:24:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: Deploying daemon mds.b on smithi044 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.37:6840/1127100718,v1:172.21.15.37:6841/1127100718] up:boot 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: fsmap 11 up:standby 2024-04-25T13:24:05.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:04 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: Deploying daemon mds.b on smithi044 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.37:6840/1127100718,v1:172.21.15.37:6841/1127100718] up:boot 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: fsmap 11 up:standby 2024-04-25T13:24:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:04 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.251+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a3c062b50 con 0x7f6a3c105360 2024-04-25T13:24:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.251+0000 7f6a397fa640 1 -- 172.21.15.37:0/3785035709 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f6a2c09c1c0 con 0x7f6a3c105360 2024-04-25T13:24:05.252 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:05.252 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:24:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-25T13:24:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 msgr2=0x7f6a240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 0x7f6a240a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000c00 tx=0x7f6a300314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 msgr2=0x7f6a3c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d770 tx=0x7f6a2c00dc30 comp rx=0 tx=0).stop 2024-04-25T13:24:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 shutdown_connections 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a3c107c90 0x7f6a3c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a240a3c30 0x7f6a240a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c105360 0x7f6a3c198710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 --2- 172.21.15.37:0/3785035709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a3c102a30 0x7f6a3c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 >> 172.21.15.37:0/3785035709 conn(0x7f6a3c0fc420 msgr2=0x7f6a3c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 shutdown_connections 2024-04-25T13:24:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:05.254+0000 7f6a42423640 1 -- 172.21.15.37:0/3785035709 wait complete. 2024-04-25T13:24:06.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:05 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3785035709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:06.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:05 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3785035709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:06.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:05 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3785035709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:06.779 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:24:06 smithi044 systemd[1]: Started Ceph mds.b for d766aeb0-0305-11ef-bc93-c7b262605968. 2024-04-25T13:24:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:06 smithi037 ceph-mon[29204]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:06 smithi069 ceph-mon[32534]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:06 smithi044 ceph-mon[32590]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.265 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:24:07.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 -- 172.21.15.37:0/4190828357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910100e60 msgr2=0x7f7910101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 --2- 172.21.15.37:0/4190828357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910100e60 0x7f7910101280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7904009900 tx=0x7f790402a410 comp rx=0 tx=0).stop 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 -- 172.21.15.37:0/4190828357 shutdown_connections 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 --2- 172.21.15.37:0/4190828357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910102a00 0x7f7910109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 --2- 172.21.15.37:0/4190828357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 0x7f79101024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 --2- 172.21.15.37:0/4190828357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910100e60 0x7f7910101280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 -- 172.21.15.37:0/4190828357 >> 172.21.15.37:0/4190828357 conn(0x7f79100fc420 msgr2=0x7f79100fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 -- 172.21.15.37:0/4190828357 shutdown_connections 2024-04-25T13:24:07.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.368+0000 7f7914cdc640 1 -- 172.21.15.37:0/4190828357 wait complete. 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 Processor -- start 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 -- start start 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 0x7f7910104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 0x7f79101035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f791010c9f0 con 0x7f7910102a00 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f791010c870 con 0x7f7910100e60 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f7914cdc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f791010cb70 con 0x7f7910102040 2024-04-25T13:24:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41820/0 (socket says 172.21.15.37:41820) 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f790dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 0x7f79101035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f790e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 0x7f7910104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.369+0000 7f790ed9d640 1 -- 172.21.15.37:0/3529192214 learned_addr learned my addr 172.21.15.37:0/3529192214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 -- 172.21.15.37:0/3529192214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 msgr2=0x7f79101035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 0x7f79101035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 -- 172.21.15.37:0/3529192214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 msgr2=0x7f7910104fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 0x7f7910104fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 -- 172.21.15.37:0/3529192214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101043c0 con 0x7f7910102a00 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790e59c640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 0x7f7910104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f790ed9d640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7908009fd0 tx=0x7f790800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908017070 con 0x7f7910102a00 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a22f0 con 0x7f7910102a00 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908003e20 con 0x7f7910102a00 2024-04-25T13:24:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080106c0 con 0x7f7910102a00 2024-04-25T13:24:07.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.370+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a2840 con 0x7f7910102a00 2024-04-25T13:24:07.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.371+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79101a2b00 con 0x7f7910102a00 2024-04-25T13:24:07.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.373+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7908004140 con 0x7f7910102a00 2024-04-25T13:24:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.374+0000 7f78ff7fe640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 0x7f78f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.374+0000 7f790e59c640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 0x7f78f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.374+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f79080cfc00 con 0x7f7910102a00 2024-04-25T13:24:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.374+0000 7f790e59c640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 0x7f78f80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7904009230 tx=0x7f79040314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:07.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.383+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79080047e0 con 0x7f7910102a00 2024-04-25T13:24:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.624+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79101a2cd0 con 0x7f7910102a00 2024-04-25T13:24:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.625+0000 7f78ff7fe640 1 -- 172.21.15.37:0/3529192214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10467 (secure 0 0 0) 0x7f790809c390 con 0x7f7910102a00 2024-04-25T13:24:07.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:07.626 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T13:24:07.626 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-04-25T13:24:07.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 msgr2=0x7f78f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 0x7f78f80a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7904009230 tx=0x7f79040314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 msgr2=0x7f7910103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7908009fd0 tx=0x7f790800b260 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 shutdown_connections 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f78f80a3bb0 0x7f78f80a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7910102a00 0x7f7910103b30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7910102040 0x7f79101035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 --2- 172.21.15.37:0/3529192214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7910100e60 0x7f7910104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 >> 172.21.15.37:0/3529192214 conn(0x7f79100fc420 msgr2=0x7f7910069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 shutdown_connections 2024-04-25T13:24:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.627+0000 7f7914cdc640 1 -- 172.21.15.37:0/3529192214 wait complete. 2024-04-25T13:24:07.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 -- 172.21.15.37:0/2475971257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628100e60 msgr2=0x7ff628101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 --2- 172.21.15.37:0/2475971257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628100e60 0x7ff628101280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff620009900 tx=0x7ff62002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 -- 172.21.15.37:0/2475971257 shutdown_connections 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 --2- 172.21.15.37:0/2475971257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628102a00 0x7ff628109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 --2- 172.21.15.37:0/2475971257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102040 0x7ff6281024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 --2- 172.21.15.37:0/2475971257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628100e60 0x7ff628101280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 -- 172.21.15.37:0/2475971257 >> 172.21.15.37:0/2475971257 conn(0x7ff6280fc420 msgr2=0x7ff6280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 -- 172.21.15.37:0/2475971257 shutdown_connections 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.745+0000 7ff62d944640 1 -- 172.21.15.37:0/2475971257 wait complete. 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 Processor -- start 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 -- start start 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628100e60 0x7ff628198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102a00 0x7ff62819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810c9d0 con 0x7ff628102040 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810c850 con 0x7ff628100e60 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff62d944640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff62810cb50 con 0x7ff628102a00 2024-04-25T13:24:07.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff6267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff6267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41838/0 (socket says 172.21.15.37:41838) 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.746+0000 7ff6267fc640 1 -- 172.21.15.37:0/4037088399 learned_addr learned my addr 172.21.15.37:0/4037088399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff626ffd640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628100e60 0x7ff628198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6277fe640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102a00 0x7ff62819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 -- 172.21.15.37:0/4037088399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102a00 msgr2=0x7ff62819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102a00 0x7ff62819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 -- 172.21.15.37:0/4037088399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628100e60 msgr2=0x7ff628198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628100e60 0x7ff628198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 -- 172.21.15.37:0/4037088399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62819fd60 con 0x7ff628102040 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff6267fc640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c920 tx=0x7ff61c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:07.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007c70 con 0x7ff628102040 2024-04-25T13:24:07.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff61c007e10 con 0x7ff628102040 2024-04-25T13:24:07.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62819fff0 con 0x7ff628102040 2024-04-25T13:24:07.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c018820 con 0x7ff628102040 2024-04-25T13:24:07.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.747+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a0560 con 0x7ff628102040 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.748+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6281a07f0 con 0x7ff628102040 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.749+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff61c00f450 con 0x7ff628102040 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.750+0000 7ff62c942640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 0x7ff6100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.750+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7ff61c0d0090 con 0x7ff628102040 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.750+0000 7ff626ffd640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 0x7ff6100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:07.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.750+0000 7ff626ffd640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 0x7ff6100a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff620009230 tx=0x7ff6200314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:07.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.754+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff61c014030 con 0x7ff628102040 2024-04-25T13:24:07.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.946+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6281a0ae0 con 0x7ff628102040 2024-04-25T13:24:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.947+0000 7ff62c942640 1 -- 172.21.15.37:0/4037088399 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18373 (secure 0 0 0) 0x7ff61c09ce70 con 0x7ff628102040 2024-04-25T13:24:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":85,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:23:21.894777+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 msgr2=0x7ff6100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 0x7ff6100a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff620009230 tx=0x7ff6200314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 msgr2=0x7ff628198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c920 tx=0x7ff61c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 shutdown_connections 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102a00 0x7ff62819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff6100a3c30 0x7ff6100a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff628102040 0x7ff628198680 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 --2- 172.21.15.37:0/4037088399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff628100e60 0x7ff628198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.949+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 >> 172.21.15.37:0/4037088399 conn(0x7ff6280fc420 msgr2=0x7ff628069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.950+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 shutdown_connections 2024-04-25T13:24:07.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:07.950+0000 7ff62d944640 1 -- 172.21.15.37:0/4037088399 wait complete. 2024-04-25T13:24:07.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:boot 2024-04-25T13:24:07.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: fsmap 12 up:standby 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:07.961 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:07 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3529192214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:07.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:24:07.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:boot 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: fsmap 12 up:standby 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.051 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.052 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:08.052 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:07 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3529192214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 -- 172.21.15.37:0/983844056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc1029a0 msgr2=0x7f32bc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 --2- 172.21.15.37:0/983844056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc1029a0 0x7f32bc1092c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009ca0 tx=0x7f32b402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 -- 172.21.15.37:0/983844056 shutdown_connections 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 --2- 172.21.15.37:0/983844056 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc1029a0 0x7f32bc1092c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 --2- 172.21.15.37:0/983844056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc101fe0 0x7f32bc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 --2- 172.21.15.37:0/983844056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc100e90 0x7f32bc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 -- 172.21.15.37:0/983844056 >> 172.21.15.37:0/983844056 conn(0x7f32bc0fc420 msgr2=0x7f32bc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 -- 172.21.15.37:0/983844056 shutdown_connections 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.073+0000 7f32c406b640 1 -- 172.21.15.37:0/983844056 wait complete. 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 Processor -- start 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 -- start start 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 0x7f32bc197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 0x7f32bc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10ca60 con 0x7f32bc101fe0 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10c8e0 con 0x7f32bc100e90 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c406b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc10cbe0 con 0x7f32bc1029a0 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41862/0 (socket says 172.21.15.37:41862) 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 -- 172.21.15.37:0/885488415 learned_addr learned my addr 172.21.15.37:0/885488415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c2608640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 0x7f32bc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1e07640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 0x7f32bc197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 -- 172.21.15.37:0/885488415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 msgr2=0x7f32bc19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 0x7f32bc19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 -- 172.21.15.37:0/885488415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 msgr2=0x7f32bc197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.074+0000 7f32c1606640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 0x7f32bc197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c1606640 1 -- 172.21.15.37:0/885488415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc19ed30 con 0x7f32bc101fe0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c1e07640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 0x7f32bc197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c2608640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 0x7f32bc19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c1606640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b450 tx=0x7f32ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac015020 con 0x7f32bc101fe0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc19efc0 con 0x7f32bc101fe0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ac011d40 con 0x7f32bc101fe0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00f340 con 0x7f32bc101fe0 2024-04-25T13:24:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.075+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc19f500 con 0x7f32bc101fe0 2024-04-25T13:24:08.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.076+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc108210 con 0x7f32bc101fe0 2024-04-25T13:24:08.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.077+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f32ac018020 con 0x7f32bc101fe0 2024-04-25T13:24:08.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.078+0000 7f32b2ffd640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 0x7f32a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.078+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f32ac0cffe0 con 0x7f32bc101fe0 2024-04-25T13:24:08.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.078+0000 7f32c1e07640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 0x7f32a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.078+0000 7f32c1e07640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 0x7f32a80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009b50 tx=0x7f32b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.081+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32ac01f020 con 0x7f32bc101fe0 2024-04-25T13:24:08.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:boot 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: fsmap 12 up:standby 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:07 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3529192214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.267+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32bc02d810 con 0x7f32bc101fe0 2024-04-25T13:24:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.268+0000 7f32b2ffd640 1 -- 172.21.15.37:0/885488415 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f32ac00d040 con 0x7f32bc101fe0 2024-04-25T13:24:08.268 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:24:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 msgr2=0x7f32a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 0x7f32a80a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32b8009b50 tx=0x7f32b8005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 msgr2=0x7f32bc197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b450 tx=0x7f32ac00b910 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 shutdown_connections 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32bc1029a0 0x7f32bc19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32a80a3bb0 0x7f32a80a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32bc101fe0 0x7f32bc197650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 --2- 172.21.15.37:0/885488415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32bc100e90 0x7f32bc197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 >> 172.21.15.37:0/885488415 conn(0x7f32bc0fc420 msgr2=0x7f32bc1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 shutdown_connections 2024-04-25T13:24:08.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.270+0000 7f32c406b640 1 -- 172.21.15.37:0/885488415 wait complete. 2024-04-25T13:24:08.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:24:08.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 -- 172.21.15.37:0/1476020843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc105360 msgr2=0x7f27cc107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/1476020843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc105360 0x7f27cc107750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009900 tx=0x7f27c802a410 comp rx=0 tx=0).stop 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 -- 172.21.15.37:0/1476020843 shutdown_connections 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/1476020843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc107c90 0x7f27cc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/1476020843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc105360 0x7f27cc107750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/1476020843 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 0x7f27cc104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 -- 172.21.15.37:0/1476020843 >> 172.21.15.37:0/1476020843 conn(0x7f27cc0fc420 msgr2=0x7f27cc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 -- 172.21.15.37:0/1476020843 shutdown_connections 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.386+0000 7f27d3d5d640 1 -- 172.21.15.37:0/1476020843 wait complete. 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 Processor -- start 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 -- start start 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 0x7f27cc1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc105360 0x7f27cc1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc10daf0 con 0x7f27cc107c90 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc10d970 con 0x7f27cc105360 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d3d5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc10dc70 con 0x7f27cc102a30 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41884/0 (socket says 172.21.15.37:41884) 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 -- 172.21.15.37:0/135560008 learned_addr learned my addr 172.21.15.37:0/135560008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d12f8640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc105360 0x7f27cc1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d1af9640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 0x7f27cc1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 -- 172.21.15.37:0/135560008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 msgr2=0x7f27cc1023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 0x7f27cc1023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 -- 172.21.15.37:0/135560008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc105360 msgr2=0x7f27cc1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc105360 0x7f27cc1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.387+0000 7f27d22fa640 1 -- 172.21.15.37:0/135560008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc1017b0 con 0x7f27cc107c90 2024-04-25T13:24:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27d22fa640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27c400efd0 tx=0x7f27c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4003b70 con 0x7f27cc107c90 2024-04-25T13:24:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc1a2320 con 0x7f27cc107c90 2024-04-25T13:24:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c4003d10 con 0x7f27cc107c90 2024-04-25T13:24:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c40108a0 con 0x7f27cc107c90 2024-04-25T13:24:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.388+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1a27f0 con 0x7f27cc107c90 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.389+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc068e00 con 0x7f27cc107c90 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.390+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f27c4004020 con 0x7f27cc107c90 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.390+0000 7f27c2ffd640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 0x7f27b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.391+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f27c40d03f0 con 0x7f27cc107c90 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.391+0000 7f27d1af9640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 0x7f27b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.391+0000 7f27d1af9640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 0x7f27b80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005fd0 tx=0x7f27bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.395+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27c409d270 con 0x7f27cc107c90 2024-04-25T13:24:08.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.607+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f27cc062b50 con 0x7f27cc107c90 2024-04-25T13:24:08.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.607+0000 7f27c2ffd640 1 -- 172.21.15.37:0/135560008 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7f27c409cb70 con 0x7f27cc107c90 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:24:08.608 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 msgr2=0x7f27b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 0x7f27b80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005fd0 tx=0x7f27bc00b390 comp rx=0 tx=0).stop 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 msgr2=0x7f27cc100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27c400efd0 tx=0x7f27c400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 shutdown_connections 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f27b80a3bb0 0x7f27b80a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27cc107c90 0x7f27cc100f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f27cc105360 0x7f27cc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 --2- 172.21.15.37:0/135560008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27cc102a30 0x7f27cc1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 >> 172.21.15.37:0/135560008 conn(0x7f27cc0fc420 msgr2=0x7f27cc1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:08.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 shutdown_connections 2024-04-25T13:24:08.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.609+0000 7f27d3d5d640 1 -- 172.21.15.37:0/135560008 wait complete. 2024-04-25T13:24:08.622 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:24:08.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:24:08.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3631934466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 msgr2=0x7f5a74104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3631934466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a74104ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009900 tx=0x7f5a6802a3f0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3631934466 shutdown_connections 2024-04-25T13:24:08.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3631934466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 0x7f5a7410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3631934466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a741077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3631934466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a74104ea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3631934466 >> 172.21.15.37:0/3631934466 conn(0x7f5a740fc420 msgr2=0x7f5a740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3631934466 shutdown_connections 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.743+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3631934466 wait complete. 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 Processor -- start 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 -- start start 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a7419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 0x7f5a741a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410db40 con 0x7f5a74102ab0 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410d9c0 con 0x7f5a741053e0 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a7ae49640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7410dcc0 con 0x7f5a74107d10 2024-04-25T13:24:08.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41898/0 (socket says 172.21.15.37:41898) 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 -- 172.21.15.37:0/3950390714 learned_addr learned my addr 172.21.15.37:0/3950390714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a793e6640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 0x7f5a741a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a73fff640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a7419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 -- 172.21.15.37:0/3950390714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 msgr2=0x7f5a741a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 0x7f5a741a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 -- 172.21.15.37:0/3950390714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 msgr2=0x7f5a7419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a7419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a78be5640 1 -- 172.21.15.37:0/3950390714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741a2030 con 0x7f5a74102ab0 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.744+0000 7f5a73fff640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a7419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:24:08.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.745+0000 7f5a78be5640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005ab0 tx=0x7f5a68003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.747+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803f070 con 0x7f5a74102ab0 2024-04-25T13:24:08.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.747+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a22c0 con 0x7f5a74102ab0 2024-04-25T13:24:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.747+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a2800 con 0x7f5a74102ab0 2024-04-25T13:24:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.747+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a68031c50 con 0x7f5a74102ab0 2024-04-25T13:24:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.747+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68049060 con 0x7f5a74102ab0 2024-04-25T13:24:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.748+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a580052c0 con 0x7f5a74102ab0 2024-04-25T13:24:08.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.750+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5a68042070 con 0x7f5a74102ab0 2024-04-25T13:24:08.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.750+0000 7f5a71ffb640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 0x7f5a640a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:08.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.751+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f5a680f06a0 con 0x7f5a74102ab0 2024-04-25T13:24:08.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.751+0000 7f5a73fff640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 0x7f5a640a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:08.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.751+0000 7f5a73fff640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 0x7f5a640a6170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a7419b920 tx=0x7f5a60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:08.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.755+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a680bce30 con 0x7f5a74102ab0 2024-04-25T13:24:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:08.956+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a580055b0 con 0x7f5a74102ab0 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4037088399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/885488415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/135560008' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:09.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:08 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/4037088399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/885488415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:24:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/135560008' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:24:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:09.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:08 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/4037088399' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/885488415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/135560008' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:09.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:08 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.835+0000 7f5a71ffb640 1 -- 172.21.15.37:0/3950390714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7f5a680c1130 con 0x7f5a74102ab0 2024-04-25T13:24:09.836 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T13:24:09.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 msgr2=0x7f5a640a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:09.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 0x7f5a640a6170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5a7419b920 tx=0x7f5a60005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:09.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 msgr2=0x7f5a7419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005ab0 tx=0x7f5a68003b90 comp rx=0 tx=0).stop 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 shutdown_connections 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107d10 0x7f5a741a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a741053e0 0x7f5a7419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5a640a3cb0 0x7f5a640a6170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 --2- 172.21.15.37:0/3950390714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102ab0 0x7f5a7419a410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 >> 172.21.15.37:0/3950390714 conn(0x7f5a740fc420 msgr2=0x7f5a741021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 shutdown_connections 2024-04-25T13:24:09.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.838+0000 7f5a7ae49640 1 -- 172.21.15.37:0/3950390714 wait complete. 2024-04-25T13:24:09.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:24:09.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/3559599731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48107c80 msgr2=0x7f9c4810a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:09.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/3559599731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48107c80 0x7f9c4810a0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009ca0 tx=0x7f9c4002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/3559599731 shutdown_connections 2024-04-25T13:24:09.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/3559599731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48107c80 0x7f9c4810a0d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/3559599731 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48105350 0x7f9c48107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/3559599731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48102a20 0x7f9c48104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/3559599731 >> 172.21.15.37:0/3559599731 conn(0x7f9c480fc430 msgr2=0x7f9c480fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/3559599731 shutdown_connections 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.967+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/3559599731 wait complete. 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 Processor -- start 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- start start 2024-04-25T13:24:09.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48102a20 0x7f9c481981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48107c80 0x7f9c4819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:09.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4810dae0 con 0x7f9c48105350 2024-04-25T13:24:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4810d960 con 0x7f9c48102a20 2024-04-25T13:24:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4810dc60 con 0x7f9c48107c80 2024-04-25T13:24:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49114/0 (socket says 172.21.15.37:49114) 2024-04-25T13:24:09.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 -- 172.21.15.37:0/2922279945 learned_addr learned my addr 172.21.15.37:0/2922279945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4d25e640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48102a20 0x7f9c481981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4da5f640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48107c80 0x7f9c4819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 -- 172.21.15.37:0/2922279945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48107c80 msgr2=0x7f9c4819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48107c80 0x7f9c4819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 -- 172.21.15.37:0/2922279945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48102a20 msgr2=0x7f9c481981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:09.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48102a20 0x7f9c481981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 -- 172.21.15.37:0/2922279945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4819fdd0 con 0x7f9c48105350 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4ca5d640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c970 tx=0x7f9c3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38007bb0 con 0x7f9c48105350 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c38007d50 con 0x7f9c48105350 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c481a00c0 con 0x7f9c48105350 2024-04-25T13:24:09.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800f450 con 0x7f9c48105350 2024-04-25T13:24:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.968+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c481a0610 con 0x7f9c48105350 2024-04-25T13:24:09.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.970+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c480fdf20 con 0x7f9c48105350 2024-04-25T13:24:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.971+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9c38004f20 con 0x7f9c48105350 2024-04-25T13:24:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.971+0000 7f9c3e7fc640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 0x7f9c340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.971+0000 7f9c4d25e640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 0x7f9c340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.971+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7f9c380d00f0 con 0x7f9c48105350 2024-04-25T13:24:09.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.972+0000 7f9c4d25e640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 0x7f9c340a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c440058c0 tx=0x7f9c4400a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:09.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:09.976+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c38014030 con 0x7f9c48105350 2024-04-25T13:24:10.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:09 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:24:10.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:09 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:10.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:09 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:24:10.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:09 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:10.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T13:24:10.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:09 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:24:10.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.163+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9c4802d5c0 con 0x7f9c48105350 2024-04-25T13:24:10.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.837+0000 7f9c3e7fc640 1 -- 172.21.15.37:0/2922279945 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7f9c3809ce00 con 0x7f9c48105350 2024-04-25T13:24:10.838 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T13:24:10.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 msgr2=0x7f9c340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:10.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 0x7f9c340a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c440058c0 tx=0x7f9c4400a250 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 msgr2=0x7f9c481986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c970 tx=0x7f9c3800ce30 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 shutdown_connections 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c48107c80 0x7f9c4819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9c340a3bb0 0x7f9c340a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c48105350 0x7f9c481986f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 --2- 172.21.15.37:0/2922279945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c48102a20 0x7f9c481981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 >> 172.21.15.37:0/2922279945 conn(0x7f9c480fc430 msgr2=0x7f9c48103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 shutdown_connections 2024-04-25T13:24:10.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.840+0000 7f9c4f4c2640 1 -- 172.21.15.37:0/2922279945 wait complete. 2024-04-25T13:24:10.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- 172.21.15.37:0/3126938465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 msgr2=0x7fa6ac1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/3126938465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac1012b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009900 tx=0x7fa6a802a410 comp rx=0 tx=0).stop 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- 172.21.15.37:0/3126938465 shutdown_connections 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/3126938465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/3126938465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/3126938465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac1012b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- 172.21.15.37:0/3126938465 >> 172.21.15.37:0/3126938465 conn(0x7fa6ac0fc420 msgr2=0x7fa6ac0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- 172.21.15.37:0/3126938465 shutdown_connections 2024-04-25T13:24:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- 172.21.15.37:0/3126938465 wait complete. 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 Processor -- start 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.965+0000 7fa6b38e1640 1 -- start start 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10ca30 con 0x7fa6ac100e90 2024-04-25T13:24:10.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10c8b0 con 0x7fa6ac101fe0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10cbb0 con 0x7fa6ac1029a0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49128/0 (socket says 172.21.15.37:49128) 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 -- 172.21.15.37:0/630847162 learned_addr learned my addr 172.21.15.37:0/630847162 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b1e7e640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b0e7c640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 -- 172.21.15.37:0/630847162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 msgr2=0x7fa6ac103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 -- 172.21.15.37:0/630847162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 msgr2=0x7fa6ac103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 -- 172.21.15.37:0/630847162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac104250 con 0x7fa6ac100e90 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b1e7e640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b0e7c640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b167d640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005ab0 tx=0x7fa6a8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:10.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a803f070 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac1a22f0 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a8031c50 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8035030 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.966+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a2800 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.967+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac068e00 con 0x7fa6ac100e90 2024-04-25T13:24:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.969+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa6a8042070 con 0x7fa6ac100e90 2024-04-25T13:24:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.969+0000 7fa6a27fc640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 0x7fa6980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.969+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7fa6a80f11f0 con 0x7fa6ac100e90 2024-04-25T13:24:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.969+0000 7fa6b0e7c640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 0x7fa6980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:10.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.970+0000 7fa6b0e7c640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 0x7fa6980a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac104b30 tx=0x7fa69c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:10.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:10.973+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6a80bdd80 con 0x7fa6ac100e90 2024-04-25T13:24:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:10 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:10 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3950390714' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:10 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:24:11.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.204+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa6ac1a2b20 con 0x7fa6ac100e90 2024-04-25T13:24:11.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.860+0000 7fa6a27fc640 1 -- 172.21.15.37:0/630847162 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa6a8031e00 con 0x7fa6ac100e90 2024-04-25T13:24:11.861 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 msgr2=0x7fa6980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 0x7fa6980a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac104b30 tx=0x7fa69c005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 msgr2=0x7fa6ac104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005ab0 tx=0x7fa6a8003b90 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 shutdown_connections 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6ac1029a0 0x7fa6ac103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa6ac101fe0 0x7fa6ac103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa6980a3bb0 0x7fa6980a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 --2- 172.21.15.37:0/630847162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6ac100e90 0x7fa6ac104dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 >> 172.21.15.37:0/630847162 conn(0x7fa6ac0fc420 msgr2=0x7fa6ac0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 shutdown_connections 2024-04-25T13:24:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.865+0000 7fa6b38e1640 1 -- 172.21.15.37:0/630847162 wait complete. 2024-04-25T13:24:11.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 -- 172.21.15.37:0/115734097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90105360 msgr2=0x7f8b90107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 --2- 172.21.15.37:0/115734097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90105360 0x7f8b90107750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009900 tx=0x7f8b8402a3f0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 -- 172.21.15.37:0/115734097 shutdown_connections 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 --2- 172.21.15.37:0/115734097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90107c90 0x7f8b9010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 --2- 172.21.15.37:0/115734097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90105360 0x7f8b90107750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 --2- 172.21.15.37:0/115734097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 0x7f8b90104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 -- 172.21.15.37:0/115734097 >> 172.21.15.37:0/115734097 conn(0x7f8b900fc420 msgr2=0x7f8b900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 -- 172.21.15.37:0/115734097 shutdown_connections 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 -- 172.21.15.37:0/115734097 wait complete. 2024-04-25T13:24:12.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.997+0000 7f8b9689e640 1 Processor -- start 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 -- start start 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 0x7f8b901980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90105360 0x7f8b90198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9010dab0 con 0x7f8b90107c90 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9010d930 con 0x7f8b90102a30 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b9689e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9010dc30 con 0x7f8b90105360 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b8f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90105360 0x7f8b90198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49154/0 (socket says 172.21.15.37:49154) 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 -- 172.21.15.37:0/2292540901 learned_addr learned my addr 172.21.15.37:0/2292540901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b8ffff640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 0x7f8b901980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 -- 172.21.15.37:0/2292540901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90105360 msgr2=0x7f8b90198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90105360 0x7f8b90198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 -- 172.21.15.37:0/2292540901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 msgr2=0x7f8b901980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 0x7f8b901980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 -- 172.21.15.37:0/2292540901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9019fd10 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.998+0000 7f8b94e3b640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800cce0 tx=0x7f8b8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.999+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800f450 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.999+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b8800faa0 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.999+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88018b70 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.999+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9019ffa0 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:11.999+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901a0470 con 0x7f8b90107c90 2024-04-25T13:24:12.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.000+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b900fdf10 con 0x7f8b90107c90 2024-04-25T13:24:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.001+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b88018d10 con 0x7f8b90107c90 2024-04-25T13:24:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.001+0000 7f8b8d7fa640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 0x7f8b780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.002+0000 7f8b8ffff640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 0x7f8b780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.002+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f8b880d0700 con 0x7f8b90107c90 2024-04-25T13:24:12.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.002+0000 7f8b8ffff640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 0x7f8b780a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8b80005e10 tx=0x7f8b80005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.006+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b88014030 con 0x7f8b90107c90 2024-04-25T13:24:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:11 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:24:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:11 smithi069 ceph-mon[32534]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:24:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:11 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:24:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:11 smithi069 ceph-mon[32534]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:24:12.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:11 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:24:12.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:11 smithi044 ceph-mon[32590]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:24:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:11 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:24:12.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:11 smithi044 ceph-mon[32590]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:24:12.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.196+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b90062b50 con 0x7f8b90107c90 2024-04-25T13:24:12.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.198+0000 7f8b8d7fa640 1 -- 172.21.15.37:0/2292540901 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22149 (secure 0 0 0) 0x7f8b8809d230 con 0x7f8b90107c90 2024-04-25T13:24:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:12.199 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:24:11.206127+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 msgr2=0x7f8b780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 0x7f8b780a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8b80005e10 tx=0x7f8b80005da0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 msgr2=0x7f8b9019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800cce0 tx=0x7f8b8800a620 comp rx=0 tx=0).stop 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 shutdown_connections 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8b780a3bb0 0x7f8b780a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b90107c90 0x7f8b9019f6a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b90105360 0x7f8b90198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 --2- 172.21.15.37:0/2292540901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b90102a30 0x7f8b901980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 >> 172.21.15.37:0/2292540901 conn(0x7f8b900fc420 msgr2=0x7f8b90103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 shutdown_connections 2024-04-25T13:24:12.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.200+0000 7f8b9689e640 1 -- 172.21.15.37:0/2292540901 wait complete. 2024-04-25T13:24:12.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:11 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2922279945' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T13:24:12.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:11 smithi037 ceph-mon[29204]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T13:24:12.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:11 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T13:24:12.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:11 smithi037 ceph-mon[29204]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T13:24:12.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:24:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 -- 172.21.15.37:0/1987537144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 msgr2=0x7f9e68109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/1987537144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e68109320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009ca0 tx=0x7f9e6002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 -- 172.21.15.37:0/1987537144 shutdown_connections 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/1987537144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e68109320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/1987537144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68102040 0x7f9e681024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/1987537144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68100e60 0x7f9e68101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 -- 172.21.15.37:0/1987537144 >> 172.21.15.37:0/1987537144 conn(0x7f9e680fc460 msgr2=0x7f9e680fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 -- 172.21.15.37:0/1987537144 shutdown_connections 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.313+0000 7f9e6d27c640 1 -- 172.21.15.37:0/1987537144 wait complete. 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 Processor -- start 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 -- start start 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68100e60 0x7f9e68198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 0x7f9e681988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810caf0 con 0x7f9e68102a00 2024-04-25T13:24:12.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810c970 con 0x7f9e68100e60 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e6d27c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810cc70 con 0x7f9e68102040 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49172/0 (socket says 172.21.15.37:49172) 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.314+0000 7f9e677fe640 1 -- 172.21.15.37:0/290512425 learned_addr learned my addr 172.21.15.37:0/290512425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e66ffd640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68100e60 0x7f9e68198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e667fc640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 0x7f9e681988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 -- 172.21.15.37:0/290512425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 msgr2=0x7f9e681988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 0x7f9e681988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 -- 172.21.15.37:0/290512425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68100e60 msgr2=0x7f9e68198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68100e60 0x7f9e68198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 -- 172.21.15.37:0/290512425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6819ff40 con 0x7f9e68102a00 2024-04-25T13:24:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e667fc640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 0x7f9e681988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e677fe640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e60031a20 tx=0x7f9e60003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60004240 con 0x7f9e68102a00 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e60003e10 con 0x7f9e68102a00 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e681a01d0 con 0x7f9e68102a00 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6002fd90 con 0x7f9e68102a00 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.315+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e681a07a0 con 0x7f9e68102a00 2024-04-25T13:24:12.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.316+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e681a0a00 con 0x7f9e68102a00 2024-04-25T13:24:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.318+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e60004b40 con 0x7f9e68102a00 2024-04-25T13:24:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.319+0000 7f9e4ffff640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 0x7f9e500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.319+0000 7f9e66ffd640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 0x7f9e500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.319+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9e600f1d90 con 0x7f9e68102a00 2024-04-25T13:24:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.319+0000 7f9e66ffd640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 0x7f9e500a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e68101bb0 tx=0x7f9e58009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.322+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e600be820 con 0x7f9e68102a00 2024-04-25T13:24:12.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.508+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e681a0cf0 con 0x7f9e68102a00 2024-04-25T13:24:12.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.510+0000 7f9e4ffff640 1 -- 172.21.15.37:0/290512425 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22149 (secure 0 0 0) 0x7f9e600be120 con 0x7f9e68102a00 2024-04-25T13:24:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:12.511 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:24:11.206127+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:12.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 msgr2=0x7f9e500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 0x7f9e500a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e68101bb0 tx=0x7f9e58009110 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 msgr2=0x7f9e6819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e60031a20 tx=0x7f9e60003c20 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 shutdown_connections 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9e500a3bb0 0x7f9e500a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e68102a00 0x7f9e6819f8d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102040 0x7f9e681988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 --2- 172.21.15.37:0/290512425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e68100e60 0x7f9e68198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 >> 172.21.15.37:0/290512425 conn(0x7f9e680fc460 msgr2=0x7f9e680fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 shutdown_connections 2024-04-25T13:24:12.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.512+0000 7f9e6d27c640 1 -- 172.21.15.37:0/290512425 wait complete. 2024-04-25T13:24:12.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:24:12.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 -- 172.21.15.37:0/1369473948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a441031a0 msgr2=0x7f9a44103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 --2- 172.21.15.37:0/1369473948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a441031a0 0x7f9a44103620 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009900 tx=0x7f9a2c02a410 comp rx=0 tx=0).stop 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 -- 172.21.15.37:0/1369473948 shutdown_connections 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 --2- 172.21.15.37:0/1369473948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44103b60 0x7f9a4410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 --2- 172.21.15.37:0/1369473948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a441031a0 0x7f9a44103620 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 --2- 172.21.15.37:0/1369473948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a44101fb0 0x7f9a441023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 -- 172.21.15.37:0/1369473948 >> 172.21.15.37:0/1369473948 conn(0x7f9a440fd520 msgr2=0x7f9a440ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 -- 172.21.15.37:0/1369473948 shutdown_connections 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.633+0000 7f9a49464640 1 -- 172.21.15.37:0/1369473948 wait complete. 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 Processor -- start 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 -- start start 2024-04-25T13:24:12.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 0x7f9a44198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 0x7f9a44198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410daf0 con 0x7f9a44103b60 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410d970 con 0x7f9a441031a0 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a49464640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410dc70 con 0x7f9a44101fb0 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49200/0 (socket says 172.21.15.37:49200) 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 -- 172.21.15.37:0/2968175335 learned_addr learned my addr 172.21.15.37:0/2968175335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a42ffd640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 0x7f9a44198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a3bfff640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 0x7f9a44198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 -- 172.21.15.37:0/2968175335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 msgr2=0x7f9a44198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 0x7f9a44198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 -- 172.21.15.37:0/2968175335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 msgr2=0x7f9a44198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:12.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 0x7f9a44198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 -- 172.21.15.37:0/2968175335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4419fe30 con 0x7f9a44103b60 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a42ffd640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 0x7f9a44198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a3bfff640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 0x7f9a44198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.634+0000 7f9a437fe640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00cad0 tx=0x7f9a3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.635+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00ae20 con 0x7f9a44103b60 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.635+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3c005020 con 0x7f9a44103b60 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.635+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a441a0120 con 0x7f9a44103b60 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.635+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00f6b0 con 0x7f9a44103b60 2024-04-25T13:24:12.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.635+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a0670 con 0x7f9a44103b60 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.636+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a44109370 con 0x7f9a44103b60 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.637+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a3c0051d0 con 0x7f9a44103b60 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.638+0000 7f9a40ff9640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 0x7f9a300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.638+0000 7f9a42ffd640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 0x7f9a300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.638+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f9a3c0d03a0 con 0x7f9a44103b60 2024-04-25T13:24:12.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.638+0000 7f9a42ffd640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 0x7f9a300a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a44103000 tx=0x7f9a34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:12.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.643+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a3c014030 con 0x7f9a44103b60 2024-04-25T13:24:12.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:12.873+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9a4402d5c0 con 0x7f9a44103b60 2024-04-25T13:24:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: pgmap v290: 97 pgs: 27 creating+peering, 68 unknown, 2 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:24:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:24:13.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:24:13.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:24:13.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:24:13.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2292540901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:12 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/290512425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: pgmap v290: 97 pgs: 27 creating+peering, 68 unknown, 2 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:24:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:24:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:24:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:24:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:24:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2292540901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:12 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/290512425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: pgmap v290: 97 pgs: 27 creating+peering, 68 unknown, 2 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T13:24:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T13:24:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/630847162' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T13:24:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T13:24:13.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T13:24:13.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2292540901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:12 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/290512425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:13.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.875+0000 7f9a40ff9640 1 -- 172.21.15.37:0/2968175335 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9a3c09ced0 con 0x7f9a44103b60 2024-04-25T13:24:13.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 msgr2=0x7f9a300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:13.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 0x7f9a300a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a44103000 tx=0x7f9a34005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:13.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 msgr2=0x7f9a4419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:13.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00cad0 tx=0x7f9a3c00cf90 comp rx=0 tx=0).stop 2024-04-25T13:24:13.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 shutdown_connections 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a300a3c30 0x7f9a300a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a44103b60 0x7f9a4419f7c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a441031a0 0x7f9a44198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 --2- 172.21.15.37:0/2968175335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a44101fb0 0x7f9a44198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 >> 172.21.15.37:0/2968175335 conn(0x7f9a440fd520 msgr2=0x7f9a44106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 shutdown_connections 2024-04-25T13:24:13.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:13.878+0000 7f9a49464640 1 -- 172.21.15.37:0/2968175335 wait complete. 2024-04-25T13:24:13.892 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T13:24:13.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T13:24:14.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.002+0000 7f17b6f9b640 1 -- 172.21.15.37:0/1082922290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b0100e90 msgr2=0x7f17b01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:14.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.002+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/1082922290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b0100e90 0x7f17b01012b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009900 tx=0x7f17a002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:14.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- 172.21.15.37:0/1082922290 shutdown_connections 2024-04-25T13:24:14.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/1082922290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b01029a0 0x7f17b01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/1082922290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0101fe0 0x7f17b0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/1082922290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b0100e90 0x7f17b01012b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- 172.21.15.37:0/1082922290 >> 172.21.15.37:0/1082922290 conn(0x7f17b00fc420 msgr2=0x7f17b00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- 172.21.15.37:0/1082922290 shutdown_connections 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- 172.21.15.37:0/1082922290 wait complete. 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 Processor -- start 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- start start 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0100e90 0x7f17b0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 0x7f17b0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17b010c9e0 con 0x7f17b01029a0 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f17b010c860 con 0x7f17b0101fe0 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b6f9b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17b010cb60 con 0x7f17b0100e90 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.003+0000 7f17b5538640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:14.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49216/0 (socket says 172.21.15.37:49216) 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 -- 172.21.15.37:0/2773646917 learned_addr learned my addr 172.21.15.37:0/2773646917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b4d37640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0100e90 0x7f17b0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17affff640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 0x7f17b0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 -- 172.21.15.37:0/2773646917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0100e90 msgr2=0x7f17b0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0100e90 0x7f17b0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 -- 172.21.15.37:0/2773646917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 msgr2=0x7f17b0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 0x7f17b0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 -- 172.21.15.37:0/2773646917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b019fe50 con 0x7f17b01029a0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17affff640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 0x7f17b0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b5538640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17a800eaa0 tx=0x7f17a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800cdb0 con 0x7f17b01029a0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a8005020 con 0x7f17b01029a0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b01a0140 con 0x7f17b01029a0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a80116d0 con 0x7f17b01029a0 2024-04-25T13:24:14.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.004+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b01a0690 con 0x7f17b01029a0 2024-04-25T13:24:14.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.005+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b01a0950 con 0x7f17b01029a0 2024-04-25T13:24:14.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.006+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17a80051d0 con 0x7f17b01029a0 2024-04-25T13:24:14.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.007+0000 7f17adffb640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 0x7f17980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:14.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.007+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f17a80d0440 con 0x7f17b01029a0 2024-04-25T13:24:14.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.007+0000 7f17b4d37640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 0x7f17980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:14.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.007+0000 7f17b4d37640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 0x7f17980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:14.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.012+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17a8016030 con 0x7f17b01029a0 2024-04-25T13:24:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:13 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:active 2024-04-25T13:24:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:13 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:13 smithi069 ceph-mon[32534]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:24:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:13 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:13 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:24:14.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:13 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:active 2024-04-25T13:24:14.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:13 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:13 smithi044 ceph-mon[32590]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:24:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:13 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:13 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:24:14.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.237+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f17b01a0ca0 con 0x7f17b01029a0 2024-04-25T13:24:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:13 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] up:active 2024-04-25T13:24:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:13 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:13 smithi037 ceph-mon[29204]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T13:24:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:13 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:14.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:13 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T13:24:14.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.876+0000 7f17adffb640 1 -- 172.21.15.37:0/2773646917 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f17a809cf70 con 0x7f17b01029a0 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 msgr2=0x7f17980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 0x7f17980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a00314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 msgr2=0x7f17b019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17a800eaa0 tx=0x7f17a800ef60 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 shutdown_connections 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f17980a3c30 0x7f17980a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17b01029a0 0x7f17b019f7e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17b0101fe0 0x7f17b0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 --2- 172.21.15.37:0/2773646917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17b0100e90 0x7f17b0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 >> 172.21.15.37:0/2773646917 conn(0x7f17b00fc420 msgr2=0x7f17b0069280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.879+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 shutdown_connections 2024-04-25T13:24:14.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:14.880+0000 7f17b6f9b640 1 -- 172.21.15.37:0/2773646917 wait complete. 2024-04-25T13:24:14.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 -- 172.21.15.37:0/720340697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4100860 msgr2=0x7f25b4100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 --2- 172.21.15.37:0/720340697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4100860 0x7f25b4100c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009900 tx=0x7f25a402a410 comp rx=0 tx=0).stop 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 -- 172.21.15.37:0/720340697 shutdown_connections 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 --2- 172.21.15.37:0/720340697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4101b60 0x7f25b4101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 --2- 172.21.15.37:0/720340697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 0x7f25b4101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 --2- 172.21.15.37:0/720340697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4100860 0x7f25b4100c80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 -- 172.21.15.37:0/720340697 >> 172.21.15.37:0/720340697 conn(0x7f25b40fc420 msgr2=0x7f25b40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 -- 172.21.15.37:0/720340697 shutdown_connections 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.005+0000 7f25ba079640 1 -- 172.21.15.37:0/720340697 wait complete. 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 Processor -- start 2024-04-25T13:24:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 -- start start 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 0x7f25b419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 0x7f25b419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25b410fcb0 con 0x7f25b4101b60 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25b410fb30 con 0x7f25b4100860 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25ba079640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25b410fe30 con 0x7f25b41011c0 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49230/0 (socket says 172.21.15.37:49230) 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 -- 172.21.15.37:0/4108668528 learned_addr learned my addr 172.21.15.37:0/4108668528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b2ffd640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 0x7f25b419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 -- 172.21.15.37:0/4108668528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 msgr2=0x7f25b419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b37fe640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 0x7f25b419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 0x7f25b419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 -- 172.21.15.37:0/4108668528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 msgr2=0x7f25b419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.006+0000 7f25b3fff640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 0x7f25b419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b3fff640 1 -- 172.21.15.37:0/4108668528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41a2080 con 0x7f25b4101b60 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b37fe640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 0x7f25b419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b3fff640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00cad0 tx=0x7f25ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00ae20 con 0x7f25b4101b60 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41a2370 con 0x7f25b4101b60 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25ac005020 con 0x7f25b4101b60 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00f6b0 con 0x7f25b4101b60 2024-04-25T13:24:15.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.007+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b41a28c0 con 0x7f25b4101b60 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.008+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b410b5b0 con 0x7f25b4101b60 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.009+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f25ac0051d0 con 0x7f25b4101b60 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.010+0000 7f25b0ff9640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 0x7f259c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.010+0000 7f25b37fe640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 0x7f259c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.010+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f25ac0d0450 con 0x7f25b4101b60 2024-04-25T13:24:15.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.010+0000 7f25b37fe640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 0x7f259c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006010 tx=0x7f25a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.014+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25ac014030 con 0x7f25b4101b60 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: pgmap v293: 97 pgs: 27 creating+peering, 60 unknown, 10 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: Cluster is now healthy 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:14 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:24:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: pgmap v293: 97 pgs: 27 creating+peering, 60 unknown, 10 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:24:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: Cluster is now healthy 2024-04-25T13:24:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:24:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:14 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:24:15.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.242+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f25b4062b50 con 0x7f25b4101b60 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: pgmap v293: 97 pgs: 27 creating+peering, 60 unknown, 10 active+clean; 577 KiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: Cluster is now healthy 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2968175335' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:14 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T13:24:15.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.891+0000 7f25b0ff9640 1 -- 172.21.15.37:0/4108668528 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f25ac09cee0 con 0x7f25b4101b60 2024-04-25T13:24:15.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 msgr2=0x7f259c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:15.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 0x7f259c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25a4006010 tx=0x7f25a4000bc0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 msgr2=0x7f25b41a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:15.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00cad0 tx=0x7f25ac00cf90 comp rx=0 tx=0).stop 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 shutdown_connections 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f259c0a3bb0 0x7f259c0a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b4101b60 0x7f25b41a1a10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25b41011c0 0x7f25b419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 --2- 172.21.15.37:0/4108668528 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25b4100860 0x7f25b419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 >> 172.21.15.37:0/4108668528 conn(0x7f25b40fc420 msgr2=0x7f25b40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 shutdown_connections 2024-04-25T13:24:15.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:15.895+0000 7f25ba079640 1 -- 172.21.15.37:0/4108668528 wait complete. 2024-04-25T13:24:15.909 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T13:24:15.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T13:24:16.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 -- 172.21.15.37:0/4115533014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 msgr2=0x7f3cac104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 --2- 172.21.15.37:0/4115533014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac104e70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009900 tx=0x7f3ca002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 -- 172.21.15.37:0/4115533014 shutdown_connections 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 --2- 172.21.15.37:0/4115533014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac107ce0 0x7f3cac10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 --2- 172.21.15.37:0/4115533014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac1053b0 0x7f3cac1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 --2- 172.21.15.37:0/4115533014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac104e70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 -- 172.21.15.37:0/4115533014 >> 172.21.15.37:0/4115533014 conn(0x7f3cac0fc430 msgr2=0x7f3cac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 -- 172.21.15.37:0/4115533014 shutdown_connections 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.020+0000 7f3cb3176640 1 -- 172.21.15.37:0/4115533014 wait complete. 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 Processor -- start 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 -- start start 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac1053b0 0x7f3cac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 0x7f3cac19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac10dae0 con 0x7f3cac102a80 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac10d960 con 0x7f3cac1053b0 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb3176640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac10dc60 con 0x7f3cac107ce0 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49242/0 (socket says 172.21.15.37:49242) 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 -- 172.21.15.37:0/3756070547 learned_addr learned my addr 172.21.15.37:0/3756070547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb1713640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 0x7f3cac19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cabfff640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac1053b0 0x7f3cac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 -- 172.21.15.37:0/3756070547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 msgr2=0x7f3cac19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 0x7f3cac19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 -- 172.21.15.37:0/3756070547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac1053b0 msgr2=0x7f3cac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac1053b0 0x7f3cac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb0f12640 1 -- 172.21.15.37:0/3756070547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac19fe50 con 0x7f3cac102a80 2024-04-25T13:24:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.021+0000 7f3cb1713640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 0x7f3cac19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3cb0f12640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005940 tx=0x7f3ca0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca003f030 con 0x7f3cac102a80 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac1a00e0 con 0x7f3cac102a80 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca0003cb0 con 0x7f3cac102a80 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0035070 con 0x7f3cac102a80 2024-04-25T13:24:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.022+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1a05d0 con 0x7f3cac102a80 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.023+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac0fdf20 con 0x7f3cac102a80 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.024+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ca0042030 con 0x7f3cac102a80 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.025+0000 7f3ca9ffb640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 0x7f3c9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.025+0000 7f3cabfff640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 0x7f3c9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.025+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f3ca00f0fd0 con 0x7f3cac102a80 2024-04-25T13:24:16.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.025+0000 7f3cabfff640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 0x7f3c9c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005fd0 tx=0x7f3c98005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.028+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ca00bda60 con 0x7f3cac102a80 2024-04-25T13:24:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:15 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:24:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:15 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:15 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:15 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:24:16.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:15 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:24:16.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:15 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:15 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:15 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:24:16.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.241+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3cac062390 con 0x7f3c9c0a3bb0 2024-04-25T13:24:16.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2773646917' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T13:24:16.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:15 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T13:24:16.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:15 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:16.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T13:24:16.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.262+0000 7f3ca9ffb640 1 -- 172.21.15.37:0/3756070547 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3cac062390 con 0x7f3c9c0a3bb0 2024-04-25T13:24:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 msgr2=0x7f3c9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 0x7f3c9c0a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3c98005fd0 tx=0x7f3c98005850 comp rx=0 tx=0).stop 2024-04-25T13:24:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 msgr2=0x7f3cac198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0005940 tx=0x7f3ca0004430 comp rx=0 tx=0).stop 2024-04-25T13:24:16.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 shutdown_connections 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cac107ce0 0x7f3cac19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3cac1053b0 0x7f3cac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f3c9c0a3bb0 0x7f3c9c0a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 --2- 172.21.15.37:0/3756070547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cac102a80 0x7f3cac198230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.264+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 >> 172.21.15.37:0/3756070547 conn(0x7f3cac0fc430 msgr2=0x7f3cac103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.265+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 shutdown_connections 2024-04-25T13:24:16.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.265+0000 7f3cb3176640 1 -- 172.21.15.37:0/3756070547 wait complete. 2024-04-25T13:24:16.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T13:24:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 -- 172.21.15.37:0/2829307002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 msgr2=0x7faf7c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 --2- 172.21.15.37:0/2829307002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c103620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009900 tx=0x7faf6c02a410 comp rx=0 tx=0).stop 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 -- 172.21.15.37:0/2829307002 shutdown_connections 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 --2- 172.21.15.37:0/2829307002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 0x7faf7c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 --2- 172.21.15.37:0/2829307002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c103620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 --2- 172.21.15.37:0/2829307002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 -- 172.21.15.37:0/2829307002 >> 172.21.15.37:0/2829307002 conn(0x7faf7c0fd520 msgr2=0x7faf7c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 -- 172.21.15.37:0/2829307002 shutdown_connections 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.382+0000 7faf82dfe640 1 -- 172.21.15.37:0/2829307002 wait complete. 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 Processor -- start 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 -- start start 2024-04-25T13:24:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 0x7faf7c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10dbc0 con 0x7faf7c1031a0 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10da40 con 0x7faf7c101fb0 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf82dfe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c10dd40 con 0x7faf7c103b60 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49290/0 (socket says 172.21.15.37:49290) 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 -- 172.21.15.37:0/3257896071 learned_addr learned my addr 172.21.15.37:0/3257896071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf80b9a640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf8139b640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 0x7faf7c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 -- 172.21.15.37:0/3257896071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 msgr2=0x7faf7c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 0x7faf7c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 -- 172.21.15.37:0/3257896071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 msgr2=0x7faf7c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 -- 172.21.15.37:0/3257896071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c19fe80 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf80b9a640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf73fff640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005ab0 tx=0x7faf6c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c03f070 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.383+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf6c03b460 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.384+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c1a0110 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.384+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c035030 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.384+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a06d0 con 0x7faf7c1031a0 2024-04-25T13:24:16.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.385+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c109350 con 0x7faf7c1031a0 2024-04-25T13:24:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.386+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf6c042070 con 0x7faf7c1031a0 2024-04-25T13:24:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.386+0000 7faf71ffb640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 0x7faf640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.386+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7faf6c0f1060 con 0x7faf7c1031a0 2024-04-25T13:24:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.386+0000 7faf80b9a640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 0x7faf640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.387+0000 7faf80b9a640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 0x7faf640a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf7c103000 tx=0x7faf78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.390+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf6c0bdb90 con 0x7faf7c1031a0 2024-04-25T13:24:16.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:24:16.246+0000 7fedd35fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:24:16.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:24:16.246+0000 7fedd35fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:24:16.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:24:16.246+0000 7fedd35fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:24:16.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:24:16.246+0000 7fedd35fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:24:16.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:24:16.246+0000 7fedd35fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:24:16.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.606+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faf7c062420 con 0x7faf640a3c30 2024-04-25T13:24:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.632+0000 7faf71ffb640 1 -- 172.21.15.37:0/3257896071 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf7c062420 con 0x7faf640a3c30 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 msgr2=0x7faf640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 0x7faf640a60f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf7c103000 tx=0x7faf78008040 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 msgr2=0x7faf7c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005ab0 tx=0x7faf6c003cd0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 shutdown_connections 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf7c103b60 0x7faf7c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7faf640a3c30 0x7faf640a60f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7c1031a0 0x7faf7c1987a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 --2- 172.21.15.37:0/3257896071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf7c101fb0 0x7faf7c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 >> 172.21.15.37:0/3257896071 conn(0x7faf7c0fd520 msgr2=0x7faf7c106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 shutdown_connections 2024-04-25T13:24:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.634+0000 7faf82dfe640 1 -- 172.21.15.37:0/3257896071 wait complete. 2024-04-25T13:24:16.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T13:24:16.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 -- 172.21.15.37:0/4211214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4105350 msgr2=0x7ff3f4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/4211214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4105350 0x7ff3f4107740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4009900 tx=0x7ff3e402a410 comp rx=0 tx=0).stop 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 -- 172.21.15.37:0/4211214941 shutdown_connections 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/4211214941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 0x7ff3f410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/4211214941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4105350 0x7ff3f4107740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/4211214941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4102a20 0x7ff3f4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 -- 172.21.15.37:0/4211214941 >> 172.21.15.37:0/4211214941 conn(0x7ff3f40fc430 msgr2=0x7ff3f40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 -- 172.21.15.37:0/4211214941 shutdown_connections 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 -- 172.21.15.37:0/4211214941 wait complete. 2024-04-25T13:24:16.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.751+0000 7ff3f94cb640 1 Processor -- start 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 -- start start 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 0x7ff3f4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 0x7ff3f419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f410dae0 con 0x7ff3f4102a20 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f410d960 con 0x7ff3f4105350 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f94cb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f410dc60 con 0x7ff3f4107c80 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49304/0 (socket says 172.21.15.37:49304) 2024-04-25T13:24:16.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/2801223958 learned_addr learned my addr 172.21.15.37:0/2801223958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f37fe640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 0x7ff3f419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f27fc640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 0x7ff3f4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/2801223958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 msgr2=0x7ff3f419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 0x7ff3f419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/2801223958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 msgr2=0x7ff3f4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 0x7ff3f4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 -- 172.21.15.37:0/2801223958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f419fdf0 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f27fc640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 0x7ff3f4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3f2ffd640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d950 tx=0x7ff3e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.752+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8017070 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.753+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f41a00e0 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.753+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e8009c30 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.753+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800f620 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.753+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a0650 con 0x7ff3f4102a20 2024-04-25T13:24:16.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.753+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f40fdf20 con 0x7ff3f4102a20 2024-04-25T13:24:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.755+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff3e8010460 con 0x7ff3f4102a20 2024-04-25T13:24:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.755+0000 7ff3dbfff640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 0x7ff3dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.755+0000 7ff3f27fc640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 0x7ff3dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.755+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7ff3e80cfec0 con 0x7ff3f4102a20 2024-04-25T13:24:16.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.756+0000 7ff3f27fc640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 0x7ff3dc0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:16.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.759+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3e809c9f0 con 0x7ff3f4102a20 2024-04-25T13:24:16.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.976+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff3f4062420 con 0x7ff3dc0a3bb0 2024-04-25T13:24:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.982+0000 7ff3dbfff640 1 -- 172.21.15.37:0/2801223958 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff3f4062420 con 0x7ff3dc0a3bb0 2024-04-25T13:24:16.982 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/3cc0f38a-4f8e-4c6f-9a02-fa2970c6f5e4 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.983+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 msgr2=0x7ff3dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.983+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 0x7ff3dc0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4000c00 tx=0x7ff3e40314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.983+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 msgr2=0x7ff3f41981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.983+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800d950 tx=0x7ff3e800de10 comp rx=0 tx=0).stop 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 shutdown_connections 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3f4107c80 0x7ff3f419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff3f4105350 0x7ff3f4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff3dc0a3bb0 0x7ff3dc0a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 --2- 172.21.15.37:0/2801223958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3f4102a20 0x7ff3f41981d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 >> 172.21.15.37:0/2801223958 conn(0x7ff3f40fc430 msgr2=0x7ff3f41032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 shutdown_connections 2024-04-25T13:24:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:16.984+0000 7ff3f94cb640 1 -- 172.21.15.37:0/2801223958 wait complete. 2024-04-25T13:24:16.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-mon[29204]: pgmap v294: 97 pgs: 9 creating+peering, 2 unknown, 86 active+clean; 578 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:24:16.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:24:16.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-mon[29204]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:16.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:16 smithi037 ceph-mon[29204]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:16.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.101+0000 7f2da05cc640 1 -- 172.21.15.37:0/3883334478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d981029a0 msgr2=0x7f2d981092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.101+0000 7f2da05cc640 1 --2- 172.21.15.37:0/3883334478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d981029a0 0x7f2d981092c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009ca0 tx=0x7f2d9002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- 172.21.15.37:0/3883334478 shutdown_connections 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- 172.21.15.37:0/3883334478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d981029a0 0x7f2d981092c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- 172.21.15.37:0/3883334478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d98101fe0 0x7f2d98102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- 172.21.15.37:0/3883334478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 0x7f2d981012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- 172.21.15.37:0/3883334478 >> 172.21.15.37:0/3883334478 conn(0x7f2d980fc420 msgr2=0x7f2d980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- 172.21.15.37:0/3883334478 shutdown_connections 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- 172.21.15.37:0/3883334478 wait complete. 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 Processor -- start 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- start start 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 0x7f2d98104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d981029a0 0x7f2d98103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810c9f0 con 0x7f2d98101fe0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810c870 con 0x7f2d981029a0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2da05cc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810cb70 con 0x7f2d98100e90 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2d9db67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2d9db67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49316/0 (socket says 172.21.15.37:49316) 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2d9db67640 1 -- 172.21.15.37:0/324342871 learned_addr learned my addr 172.21.15.37:0/324342871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.102+0000 7f2d9e368640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 0x7f2d98104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9eb69640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d981029a0 0x7f2d98103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 -- 172.21.15.37:0/324342871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 msgr2=0x7f2d98104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 0x7f2d98104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 -- 172.21.15.37:0/324342871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d981029a0 msgr2=0x7f2d98103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d981029a0 0x7f2d98103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 -- 172.21.15.37:0/324342871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d98104220 con 0x7f2d98101fe0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d9db67640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800d770 tx=0x7f2d8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88017070 con 0x7f2d98101fe0 2024-04-25T13:24:17.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d88009c30 con 0x7f2d98101fe0 2024-04-25T13:24:17.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a22b0 con 0x7f2d98101fe0 2024-04-25T13:24:17.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88003cc0 con 0x7f2d98101fe0 2024-04-25T13:24:17.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.103+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a2800 con 0x7f2d98101fe0 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.104+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d981a2b20 con 0x7f2d98101fe0 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.105+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2d88003e60 con 0x7f2d98101fe0 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.106+0000 7f2d8f7fe640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 0x7f2d840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.106+0000 7f2d9e368640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 0x7f2d840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.106+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f2d880cfc40 con 0x7f2d98101fe0 2024-04-25T13:24:17.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.106+0000 7f2d9e368640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 0x7f2d840a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009bd0 tx=0x7f2d94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.110+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d8809c770 con 0x7f2d98101fe0 2024-04-25T13:24:17.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:16 smithi044 ceph-mon[32590]: pgmap v294: 97 pgs: 9 creating+peering, 2 unknown, 86 active+clean; 578 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:24:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:16 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:24:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:16 smithi044 ceph-mon[32590]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:16 smithi044 ceph-mon[32590]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:17.140 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:24:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: 2024-04-25T13:24:16.785+0000 7f8142c2a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:24:17.309 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:16 smithi069 ceph-mon[32534]: pgmap v294: 97 pgs: 9 creating+peering, 2 unknown, 86 active+clean; 578 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-25T13:24:17.310 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:16 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/4108668528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T13:24:17.310 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:16 smithi069 ceph-mon[32534]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T13:24:17.310 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:16 smithi069 ceph-mon[32534]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.325+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2d980fe5f0 con 0x7f2d840a3bb0 2024-04-25T13:24:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.351+0000 7f2d8f7fe640 1 -- 172.21.15.37:0/324342871 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d980fe5f0 con 0x7f2d840a3bb0 2024-04-25T13:24:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 msgr2=0x7f2d840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 0x7f2d840a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009bd0 tx=0x7f2d94005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 msgr2=0x7f2d98103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800d770 tx=0x7f2d8800dc30 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 shutdown_connections 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d981029a0 0x7f2d98103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2d840a3bb0 0x7f2d840a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d98101fe0 0x7f2d98103450 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 --2- 172.21.15.37:0/324342871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d98100e90 0x7f2d98104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 >> 172.21.15.37:0/324342871 conn(0x7f2d980fc420 msgr2=0x7f2d980fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 shutdown_connections 2024-04-25T13:24:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.353+0000 7f2da05cc640 1 -- 172.21.15.37:0/324342871 wait complete. 2024-04-25T13:24:17.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 -- 172.21.15.37:0/240196555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4105350 msgr2=0x7f2be4107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/240196555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4105350 0x7f2be4107740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009900 tx=0x7f2be002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 -- 172.21.15.37:0/240196555 shutdown_connections 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/240196555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4107c80 0x7f2be410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/240196555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4105350 0x7f2be4107740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/240196555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 0x7f2be4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 -- 172.21.15.37:0/240196555 >> 172.21.15.37:0/240196555 conn(0x7f2be40fc430 msgr2=0x7f2be40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 -- 172.21.15.37:0/240196555 shutdown_connections 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.471+0000 7f2beb2e1640 1 -- 172.21.15.37:0/240196555 wait complete. 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 Processor -- start 2024-04-25T13:24:17.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 -- start start 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 0x7f2be4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4105350 0x7f2be41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410dae0 con 0x7f2be4107c80 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410d960 con 0x7f2be4102a20 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2beb2e1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2be410dc60 con 0x7f2be4105350 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2be987e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2be987e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49328/0 (socket says 172.21.15.37:49328) 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2be987e640 1 -- 172.21.15.37:0/3907778956 learned_addr learned my addr 172.21.15.37:0/3907778956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.472+0000 7f2be887c640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4105350 0x7f2be41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be907d640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 0x7f2be4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 -- 172.21.15.37:0/3907778956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4105350 msgr2=0x7f2be41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4105350 0x7f2be41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 -- 172.21.15.37:0/3907778956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 msgr2=0x7f2be4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 0x7f2be4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 -- 172.21.15.37:0/3907778956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be419fe30 con 0x7f2be4107c80 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2be987e640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00eaa0 tx=0x7f2bdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00cdb0 con 0x7f2be4107c80 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bdc005020 con 0x7f2be4107c80 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be41a0120 con 0x7f2be4107c80 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0116d0 con 0x7f2be4107c80 2024-04-25T13:24:17.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.473+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a0640 con 0x7f2be4107c80 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.474+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be40fdf20 con 0x7f2be4107c80 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.475+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bdc0051d0 con 0x7f2be4107c80 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.476+0000 7f2bda7fc640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 0x7f2bd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.476+0000 7f2be907d640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 0x7f2bd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.476+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f2bdc0d03c0 con 0x7f2be4107c80 2024-04-25T13:24:17.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.476+0000 7f2be907d640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 0x7f2bd00a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400bbe0 tx=0x7f2bd400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.480+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bdc016030 con 0x7f2be4107c80 2024-04-25T13:24:17.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.698+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2be4062390 con 0x7f2bd00a3bb0 2024-04-25T13:24:17.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.702+0000 7f2bda7fc640 1 -- 172.21.15.37:0/3907778956 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2be4062390 con 0x7f2bd00a3bb0 2024-04-25T13:24:17.703 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/8a9db0d3-907a-4696-b98e-19a60db0c1d3 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 msgr2=0x7f2bd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 0x7f2bd00a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400bbe0 tx=0x7f2bd400b390 comp rx=0 tx=0).stop 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 msgr2=0x7f2be419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00eaa0 tx=0x7f2bdc00ef60 comp rx=0 tx=0).stop 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 shutdown_connections 2024-04-25T13:24:17.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f2bd00a3bb0 0x7f2bd00a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be4107c80 0x7f2be419f7c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be4105350 0x7f2be41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 --2- 172.21.15.37:0/3907778956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2be4102a20 0x7f2be4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.704+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 >> 172.21.15.37:0/3907778956 conn(0x7f2be40fc430 msgr2=0x7f2be41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.705+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 shutdown_connections 2024-04-25T13:24:17.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.705+0000 7f2beb2e1640 1 -- 172.21.15.37:0/3907778956 wait complete. 2024-04-25T13:24:17.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:24:17.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 -- 172.21.15.37:0/2273684412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0102ad0 msgr2=0x7f03e0104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 --2- 172.21.15.37:0/2273684412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0102ad0 0x7f03e0104ec0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 -- 172.21.15.37:0/2273684412 shutdown_connections 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 --2- 172.21.15.37:0/2273684412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0107d30 0x7f03e010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 --2- 172.21.15.37:0/2273684412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 0x7f03e01077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 --2- 172.21.15.37:0/2273684412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0102ad0 0x7f03e0104ec0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 -- 172.21.15.37:0/2273684412 >> 172.21.15.37:0/2273684412 conn(0x7f03e00fc460 msgr2=0x7f03e00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 -- 172.21.15.37:0/2273684412 shutdown_connections 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.817+0000 7f03e6405640 1 -- 172.21.15.37:0/2273684412 wait complete. 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 Processor -- start 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 -- start start 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0102ad0 0x7f03e0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 0x7f03e0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010db10 con 0x7f03e0107d30 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010d990 con 0x7f03e0105400 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e6405640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03e010dc90 con 0x7f03e0102ad0 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e49a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e49a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49352/0 (socket says 172.21.15.37:49352) 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03e49a2640 1 -- 172.21.15.37:0/831991281 learned_addr learned my addr 172.21.15.37:0/831991281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03dffff640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0102ad0 0x7f03e0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.818+0000 7f03df7fe640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 0x7f03e0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 -- 172.21.15.37:0/831991281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0102ad0 msgr2=0x7f03e0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0102ad0 0x7f03e0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 -- 172.21.15.37:0/831991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 msgr2=0x7f03e0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 0x7f03e0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 -- 172.21.15.37:0/831991281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e019fe20 con 0x7f03e0107d30 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e49a2640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f03d800eb10 tx=0x7f03d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8004b20 con 0x7f03e0107d30 2024-04-25T13:24:17.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01a0110 con 0x7f03e0107d30 2024-04-25T13:24:17.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d8004cc0 con 0x7f03e0107d30 2024-04-25T13:24:17.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8010690 con 0x7f03e0107d30 2024-04-25T13:24:17.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.819+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01a0660 con 0x7f03e0107d30 2024-04-25T13:24:17.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.820+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e00fdf50 con 0x7f03e0107d30 2024-04-25T13:24:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.821+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f03d8005000 con 0x7f03e0107d30 2024-04-25T13:24:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.822+0000 7f03dd7fa640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.822+0000 7f03dffff640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.822+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f03d80d0970 con 0x7f03e0107d30 2024-04-25T13:24:17.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.822+0000 7f03dffff640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03d0000c00 tx=0x7f03d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:17.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.826+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03d8053ff0 con 0x7f03e0107d30 2024-04-25T13:24:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:17.910+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f03d8099410 con 0x7f03e0107d30 2024-04-25T13:24:18.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.057+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e002d5c0 con 0x7f03e0107d30 2024-04-25T13:24:18.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.058+0000 7f03dd7fa640 1 -- 172.21.15.37:0/831991281 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7f03d809d4a0 con 0x7f03e0107d30 2024-04-25T13:24:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:18.058 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:24:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:24:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 msgr2=0x7f03cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03d0000c00 tx=0x7f03d00314a0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 msgr2=0x7f03e019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f03d800eb10 tx=0x7f03d800c2d0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 shutdown_connections 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e0107d30 0x7f03e019f7b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f03e0105400 0x7f03e0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 --2- 172.21.15.37:0/831991281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03e0102ad0 0x7f03e0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 >> 172.21.15.37:0/831991281 conn(0x7f03e00fc460 msgr2=0x7f03e0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 shutdown_connections 2024-04-25T13:24:18.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.060+0000 7f03e6405640 1 -- 172.21.15.37:0/831991281 wait complete. 2024-04-25T13:24:18.070 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:17 smithi037 ceph-mon[29204]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.070 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:17 smithi037 ceph-mon[29204]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.071 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:17 smithi037 ceph-mon[29204]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- 172.21.15.37:0/620977979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 msgr2=0x7f69e4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 --2- 172.21.15.37:0/620977979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e4104ee0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009900 tx=0x7f69e002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- 172.21.15.37:0/620977979 shutdown_connections 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 --2- 172.21.15.37:0/620977979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4107d50 0x7f69e410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 --2- 172.21.15.37:0/620977979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4105420 0x7f69e4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 --2- 172.21.15.37:0/620977979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e4104ee0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- 172.21.15.37:0/620977979 >> 172.21.15.37:0/620977979 conn(0x7f69e40fc460 msgr2=0x7f69e40fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- 172.21.15.37:0/620977979 shutdown_connections 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- 172.21.15.37:0/620977979 wait complete. 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 Processor -- start 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.173+0000 7f69eb691640 1 -- start start 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4105420 0x7f69e41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 0x7f69e419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69e410db80 con 0x7f69e4102af0 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f69e410da00 con 0x7f69e4107d50 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f69e410dd00 con 0x7f69e4105420 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49368/0 (socket says 172.21.15.37:49368) 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 -- 172.21.15.37:0/2357652895 learned_addr learned my addr 172.21.15.37:0/2357652895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e8c2c640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4105420 0x7f69e41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e9c2e640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 0x7f69e419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 -- 172.21.15.37:0/2357652895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4105420 msgr2=0x7f69e41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4105420 0x7f69e41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 -- 172.21.15.37:0/2357652895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 msgr2=0x7f69e419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 0x7f69e419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 -- 172.21.15.37:0/2357652895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e419fed0 con 0x7f69e4102af0 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e9c2e640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 0x7f69e419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69e942d640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005940 tx=0x7f69e0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e003f030 con 0x7f69e4102af0 2024-04-25T13:24:18.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e0003aa0 con 0x7f69e4102af0 2024-04-25T13:24:18.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41a0160 con 0x7f69e4102af0 2024-04-25T13:24:18.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0035070 con 0x7f69e4102af0 2024-04-25T13:24:18.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.174+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e41a06d0 con 0x7f69e4102af0 2024-04-25T13:24:18.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.175+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e40fdf50 con 0x7f69e4102af0 2024-04-25T13:24:18.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.177+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69e0042030 con 0x7f69e4102af0 2024-04-25T13:24:18.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.178+0000 7f69da7fc640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.178+0000 7f69e8c2c640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.178+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f69e00f1170 con 0x7f69e4102af0 2024-04-25T13:24:18.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.178+0000 7f69e8c2c640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 0x7f69d00a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69e41997c0 tx=0x7f69d4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.182+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69e00bdaa0 con 0x7f69e4102af0 2024-04-25T13:24:18.238 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:17 smithi069 ceph-mon[32534]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.238 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:17 smithi069 ceph-mon[32534]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.238 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:17 smithi069 ceph-mon[32534]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.372+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69e4062b50 con 0x7f69e4102af0 2024-04-25T13:24:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.373+0000 7f69da7fc640 1 -- 172.21.15.37:0/2357652895 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22136 (secure 0 0 0) 0x7f69e00bd3a0 con 0x7f69e4102af0 2024-04-25T13:24:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:18.374 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:24:12.861871+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 msgr2=0x7f69d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 0x7f69d00a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f69e41997c0 tx=0x7f69d4009110 comp rx=0 tx=0).stop 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 msgr2=0x7f69e41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005940 tx=0x7f69e0031a50 comp rx=0 tx=0).stop 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 shutdown_connections 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f69e4107d50 0x7f69e419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69e4105420 0x7f69e41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69d00a3c40 0x7f69d00a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 --2- 172.21.15.37:0/2357652895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69e4102af0 0x7f69e41982b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 >> 172.21.15.37:0/2357652895 conn(0x7f69e40fc460 msgr2=0x7f69e4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 shutdown_connections 2024-04-25T13:24:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.376+0000 7f69eb691640 1 -- 172.21.15.37:0/2357652895 wait complete. 2024-04-25T13:24:18.388 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:24:18.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:24:18.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:17 smithi044 ceph-mon[32590]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:17 smithi044 ceph-mon[32590]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:17 smithi044 ceph-mon[32590]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 -- 172.21.15.37:0/1604463424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 msgr2=0x7f50ec10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 --2- 172.21.15.37:0/1604463424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec10a0e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009ca0 tx=0x7f50e402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 -- 172.21.15.37:0/1604463424 shutdown_connections 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 --2- 172.21.15.37:0/1604463424 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec10a0e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 --2- 172.21.15.37:0/1604463424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec105360 0x7f50ec107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 --2- 172.21.15.37:0/1604463424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec102a30 0x7f50ec104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 -- 172.21.15.37:0/1604463424 >> 172.21.15.37:0/1604463424 conn(0x7f50ec0fc460 msgr2=0x7f50ec0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 -- 172.21.15.37:0/1604463424 shutdown_connections 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.489+0000 7f50f0c69640 1 -- 172.21.15.37:0/1604463424 wait complete. 2024-04-25T13:24:18.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 Processor -- start 2024-04-25T13:24:18.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 -- start start 2024-04-25T13:24:18.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec102a30 0x7f50ec1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec105360 0x7f50ec198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec10daf0 con 0x7f50ec107c90 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec10d970 con 0x7f50ec102a30 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50f0c69640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec10dc70 con 0x7f50ec105360 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49384/0 (socket says 172.21.15.37:49384) 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 -- 172.21.15.37:0/231039981 learned_addr learned my addr 172.21.15.37:0/231039981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50e1d9b640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec105360 0x7f50ec198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ea59c640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec102a30 0x7f50ec1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 -- 172.21.15.37:0/231039981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec105360 msgr2=0x7f50ec198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec105360 0x7f50ec198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 -- 172.21.15.37:0/231039981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec102a30 msgr2=0x7f50ec1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec102a30 0x7f50ec1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.490+0000 7f50ead9d640 1 -- 172.21.15.37:0/231039981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec19fce0 con 0x7f50ec107c90 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50ead9d640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097b0 tx=0x7f50e4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003bf0 con 0x7f50ec107c90 2024-04-25T13:24:18.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4003d90 con 0x7f50ec107c90 2024-04-25T13:24:18.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec19ff70 con 0x7f50ec107c90 2024-04-25T13:24:18.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e402fc70 con 0x7f50ec107c90 2024-04-25T13:24:18.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.491+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec1a0430 con 0x7f50ec107c90 2024-04-25T13:24:18.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.492+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ec0fdf50 con 0x7f50ec107c90 2024-04-25T13:24:18.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.493+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50e4004040 con 0x7f50ec107c90 2024-04-25T13:24:18.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.494+0000 7f50e3fff640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 0x7f50d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.494+0000 7f50ea59c640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 0x7f50d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.494+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f50e40f1a80 con 0x7f50ec107c90 2024-04-25T13:24:18.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.494+0000 7f50ea59c640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 0x7f50d80a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009b60 tx=0x7f50dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.498+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50e40f3e20 con 0x7f50ec107c90 2024-04-25T13:24:18.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.682+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50ec062390 con 0x7f50d80a3c40 2024-04-25T13:24:18.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.690+0000 7f50e3fff640 1 -- 172.21.15.37:0/231039981 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258724 (secure 0 0 0) 0x7f50ec062390 con 0x7f50d80a3c40 2024-04-25T13:24:18.693 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:18.695 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T13:24:18.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.693+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 msgr2=0x7f50d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.693+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 0x7f50d80a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009b60 tx=0x7f50dc005f90 comp rx=0 tx=0).stop 2024-04-25T13:24:18.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 msgr2=0x7f50ec19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097b0 tx=0x7f50e4031840 comp rx=0 tx=0).stop 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 shutdown_connections 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f50d80a3c40 0x7f50d80a6100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50ec107c90 0x7f50ec19f670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50ec105360 0x7f50ec198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 --2- 172.21.15.37:0/231039981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50ec102a30 0x7f50ec1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 >> 172.21.15.37:0/231039981 conn(0x7f50ec0fc460 msgr2=0x7f50ec103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 shutdown_connections 2024-04-25T13:24:18.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.694+0000 7f50f0c69640 1 -- 172.21.15.37:0/231039981 wait complete. 2024-04-25T13:24:18.705 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-25T13:24:17.730144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":145,"num_read_kb":127,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":408044,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124533780,"statfs":{"total":1151940427776,"available":1151522590720,"internally_reserved":0,"allocated":13332480,"data_stored":7256940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":404272442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.890751"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031512+0000","last_change":"2024-04-25T13:24:11.873737+0000","last_active":"2024-04-25T13:24:13.031512+0000","last_peered":"2024-04-25T13:24:13.031512+0000","last_clean":"2024-04-25T13:24:13.031512+0000","last_became_active":"2024-04-25T13:24:11.872598+0000","last_became_peered":"2024-04-25T13:24:11.872598+0000","last_unstale":"2024-04-25T13:24:13.031512+0000","last_undegraded":"2024-04-25T13:24:13.031512+0000","last_fullsized":"2024-04-25T13:24:13.031512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:46.739869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054856+0000","last_change":"2024-04-25T13:24:11.886834+0000","last_active":"2024-04-25T13:24:13.054856+0000","last_peered":"2024-04-25T13:24:13.054856+0000","last_clean":"2024-04-25T13:24:13.054856+0000","last_became_active":"2024-04-25T13:24:11.886460+0000","last_became_peered":"2024-04-25T13:24:11.886460+0000","last_unstale":"2024-04-25T13:24:13.054856+0000","last_undegraded":"2024-04-25T13:24:13.054856+0000","last_fullsized":"2024-04-25T13:24:13.054856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.103106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054721+0000","last_change":"2024-04-25T13:24:11.877582+0000","last_active":"2024-04-25T13:24:13.054721+0000","last_peered":"2024-04-25T13:24:13.054721+0000","last_clean":"2024-04-25T13:24:13.054721+0000","last_became_active":"2024-04-25T13:24:11.877245+0000","last_became_peered":"2024-04-25T13:24:11.877245+0000","last_unstale":"2024-04-25T13:24:13.054721+0000","last_undegraded":"2024-04-25T13:24:13.054721+0000","last_fullsized":"2024-04-25T13:24:13.054721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:05.065255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031844+0000","last_change":"2024-04-25T13:24:11.873522+0000","last_active":"2024-04-25T13:24:13.031844+0000","last_peered":"2024-04-25T13:24:13.031844+0000","last_clean":"2024-04-25T13:24:13.031844+0000","last_became_active":"2024-04-25T13:24:11.872505+0000","last_became_peered":"2024-04-25T13:24:11.872505+0000","last_unstale":"2024-04-25T13:24:13.031844+0000","last_undegraded":"2024-04-25T13:24:13.031844+0000","last_fullsized":"2024-04-25T13:24:13.031844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:15.968935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876985+0000","last_change":"2024-04-25T13:24:11.873353+0000","last_active":"2024-04-25T13:24:12.876985+0000","last_peered":"2024-04-25T13:24:12.876985+0000","last_clean":"2024-04-25T13:24:12.876985+0000","last_became_active":"2024-04-25T13:24:11.872997+0000","last_became_peered":"2024-04-25T13:24:11.872997+0000","last_unstale":"2024-04-25T13:24:12.876985+0000","last_undegraded":"2024-04-25T13:24:12.876985+0000","last_fullsized":"2024-04-25T13:24:12.876985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:15.205105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032139+0000","last_change":"2024-04-25T13:24:11.875994+0000","last_active":"2024-04-25T13:24:13.032139+0000","last_peered":"2024-04-25T13:24:13.032139+0000","last_clean":"2024-04-25T13:24:13.032139+0000","last_became_active":"2024-04-25T13:24:11.875658+0000","last_became_peered":"2024-04-25T13:24:11.875658+0000","last_unstale":"2024-04-25T13:24:13.032139+0000","last_undegraded":"2024-04-25T13:24:13.032139+0000","last_fullsized":"2024-04-25T13:24:13.032139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:20.959715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872229+0000","last_change":"2024-04-25T13:24:11.879417+0000","last_active":"2024-04-25T13:24:12.872229+0000","last_peered":"2024-04-25T13:24:12.872229+0000","last_clean":"2024-04-25T13:24:12.872229+0000","last_became_active":"2024-04-25T13:24:11.879112+0000","last_became_peered":"2024-04-25T13:24:11.879112+0000","last_unstale":"2024-04-25T13:24:12.872229+0000","last_undegraded":"2024-04-25T13:24:12.872229+0000","last_fullsized":"2024-04-25T13:24:12.872229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:25.250253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876730+0000","last_change":"2024-04-25T13:24:11.883661+0000","last_active":"2024-04-25T13:24:12.876730+0000","last_peered":"2024-04-25T13:24:12.876730+0000","last_clean":"2024-04-25T13:24:12.876730+0000","last_became_active":"2024-04-25T13:24:11.883219+0000","last_became_peered":"2024-04-25T13:24:11.883219+0000","last_unstale":"2024-04-25T13:24:12.876730+0000","last_undegraded":"2024-04-25T13:24:12.876730+0000","last_fullsized":"2024-04-25T13:24:12.876730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:15.731727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031629+0000","last_change":"2024-04-25T13:24:11.880350+0000","last_active":"2024-04-25T13:24:13.031629+0000","last_peered":"2024-04-25T13:24:13.031629+0000","last_clean":"2024-04-25T13:24:13.031629+0000","last_became_active":"2024-04-25T13:24:11.880028+0000","last_became_peered":"2024-04-25T13:24:11.880028+0000","last_unstale":"2024-04-25T13:24:13.031629+0000","last_undegraded":"2024-04-25T13:24:13.031629+0000","last_fullsized":"2024-04-25T13:24:13.031629+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:54.832380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032641+0000","last_change":"2024-04-25T13:24:11.879180+0000","last_active":"2024-04-25T13:24:13.032641+0000","last_peered":"2024-04-25T13:24:13.032641+0000","last_clean":"2024-04-25T13:24:13.032641+0000","last_became_active":"2024-04-25T13:24:11.877612+0000","last_became_peered":"2024-04-25T13:24:11.877612+0000","last_unstale":"2024-04-25T13:24:13.032641+0000","last_undegraded":"2024-04-25T13:24:13.032641+0000","last_fullsized":"2024-04-25T13:24:13.032641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.269957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872729+0000","last_change":"2024-04-25T13:24:11.880768+0000","last_active":"2024-04-25T13:24:12.872729+0000","last_peered":"2024-04-25T13:24:12.872729+0000","last_clean":"2024-04-25T13:24:12.872729+0000","last_became_active":"2024-04-25T13:24:11.879900+0000","last_became_peered":"2024-04-25T13:24:11.879900+0000","last_unstale":"2024-04-25T13:24:12.872729+0000","last_undegraded":"2024-04-25T13:24:12.872729+0000","last_fullsized":"2024-04-25T13:24:12.872729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:11.941761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031611+0000","last_change":"2024-04-25T13:24:11.873868+0000","last_active":"2024-04-25T13:24:13.031611+0000","last_peered":"2024-04-25T13:24:13.031611+0000","last_clean":"2024-04-25T13:24:13.031611+0000","last_became_active":"2024-04-25T13:24:11.872941+0000","last_became_peered":"2024-04-25T13:24:11.872941+0000","last_unstale":"2024-04-25T13:24:13.031611+0000","last_undegraded":"2024-04-25T13:24:13.031611+0000","last_fullsized":"2024-04-25T13:24:13.031611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:15.580750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031013+0000","last_change":"2024-04-25T13:24:11.888054+0000","last_active":"2024-04-25T13:24:13.031013+0000","last_peered":"2024-04-25T13:24:13.031013+0000","last_clean":"2024-04-25T13:24:13.031013+0000","last_became_active":"2024-04-25T13:24:11.887217+0000","last_became_peered":"2024-04-25T13:24:11.887217+0000","last_unstale":"2024-04-25T13:24:13.031013+0000","last_undegraded":"2024-04-25T13:24:13.031013+0000","last_fullsized":"2024-04-25T13:24:13.031013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:08.058645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054548+0000","last_change":"2024-04-25T13:24:11.878049+0000","last_active":"2024-04-25T13:24:13.054548+0000","last_peered":"2024-04-25T13:24:13.054548+0000","last_clean":"2024-04-25T13:24:13.054548+0000","last_became_active":"2024-04-25T13:24:11.876595+0000","last_became_peered":"2024-04-25T13:24:11.876595+0000","last_unstale":"2024-04-25T13:24:13.054548+0000","last_undegraded":"2024-04-25T13:24:13.054548+0000","last_fullsized":"2024-04-25T13:24:13.054548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:36.126016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054560+0000","last_change":"2024-04-25T13:24:11.890173+0000","last_active":"2024-04-25T13:24:13.054560+0000","last_peered":"2024-04-25T13:24:13.054560+0000","last_clean":"2024-04-25T13:24:13.054560+0000","last_became_active":"2024-04-25T13:24:11.889865+0000","last_became_peered":"2024-04-25T13:24:11.889865+0000","last_unstale":"2024-04-25T13:24:13.054560+0000","last_undegraded":"2024-04-25T13:24:13.054560+0000","last_fullsized":"2024-04-25T13:24:13.054560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:36.809979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030690+0000","last_change":"2024-04-25T13:24:11.888024+0000","last_active":"2024-04-25T13:24:13.030690+0000","last_peered":"2024-04-25T13:24:13.030690+0000","last_clean":"2024-04-25T13:24:13.030690+0000","last_became_active":"2024-04-25T13:24:11.887497+0000","last_became_peered":"2024-04-25T13:24:11.887497+0000","last_unstale":"2024-04-25T13:24:13.030690+0000","last_undegraded":"2024-04-25T13:24:13.030690+0000","last_fullsized":"2024-04-25T13:24:13.030690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.887282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031702+0000","last_change":"2024-04-25T13:24:11.872532+0000","last_active":"2024-04-25T13:24:13.031702+0000","last_peered":"2024-04-25T13:24:13.031702+0000","last_clean":"2024-04-25T13:24:13.031702+0000","last_became_active":"2024-04-25T13:24:11.871047+0000","last_became_peered":"2024-04-25T13:24:11.871047+0000","last_unstale":"2024-04-25T13:24:13.031702+0000","last_undegraded":"2024-04-25T13:24:13.031702+0000","last_fullsized":"2024-04-25T13:24:13.031702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:21.818593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869606+0000","last_change":"2024-04-25T13:24:11.869003+0000","last_active":"2024-04-25T13:24:12.869606+0000","last_peered":"2024-04-25T13:24:12.869606+0000","last_clean":"2024-04-25T13:24:12.869606+0000","last_became_active":"2024-04-25T13:24:11.868639+0000","last_became_peered":"2024-04-25T13:24:11.868639+0000","last_unstale":"2024-04-25T13:24:12.869606+0000","last_undegraded":"2024-04-25T13:24:12.869606+0000","last_fullsized":"2024-04-25T13:24:12.869606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:03.366661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870511+0000","last_change":"2024-04-25T13:24:11.878947+0000","last_active":"2024-04-25T13:24:12.870511+0000","last_peered":"2024-04-25T13:24:12.870511+0000","last_clean":"2024-04-25T13:24:12.870511+0000","last_became_active":"2024-04-25T13:24:11.878344+0000","last_became_peered":"2024-04-25T13:24:11.878344+0000","last_unstale":"2024-04-25T13:24:12.870511+0000","last_undegraded":"2024-04-25T13:24:12.870511+0000","last_fullsized":"2024-04-25T13:24:12.870511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:54.653873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869560+0000","last_change":"2024-04-25T13:24:10.856180+0000","last_active":"2024-04-25T13:24:12.869560+0000","last_peered":"2024-04-25T13:24:12.869560+0000","last_clean":"2024-04-25T13:24:12.869560+0000","last_became_active":"2024-04-25T13:24:10.855746+0000","last_became_peered":"2024-04-25T13:24:10.855746+0000","last_unstale":"2024-04-25T13:24:12.869560+0000","last_undegraded":"2024-04-25T13:24:12.869560+0000","last_fullsized":"2024-04-25T13:24:12.869560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:58.312713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877118+0000","last_change":"2024-04-25T13:24:11.889286+0000","last_active":"2024-04-25T13:24:12.877118+0000","last_peered":"2024-04-25T13:24:12.877118+0000","last_clean":"2024-04-25T13:24:12.877118+0000","last_became_active":"2024-04-25T13:24:11.888950+0000","last_became_peered":"2024-04-25T13:24:11.888950+0000","last_unstale":"2024-04-25T13:24:12.877118+0000","last_undegraded":"2024-04-25T13:24:12.877118+0000","last_fullsized":"2024-04-25T13:24:12.877118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:31.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877190+0000","last_change":"2024-04-25T13:24:10.859008+0000","last_active":"2024-04-25T13:24:12.877190+0000","last_peered":"2024-04-25T13:24:12.877190+0000","last_clean":"2024-04-25T13:24:12.877190+0000","last_became_active":"2024-04-25T13:24:10.858350+0000","last_became_peered":"2024-04-25T13:24:10.858350+0000","last_unstale":"2024-04-25T13:24:12.877190+0000","last_undegraded":"2024-04-25T13:24:12.877190+0000","last_fullsized":"2024-04-25T13:24:12.877190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.048592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.055016+0000","last_change":"2024-04-25T13:24:11.879185+0000","last_active":"2024-04-25T13:24:13.055016+0000","last_peered":"2024-04-25T13:24:13.055016+0000","last_clean":"2024-04-25T13:24:13.055016+0000","last_became_active":"2024-04-25T13:24:11.878808+0000","last_became_peered":"2024-04-25T13:24:11.878808+0000","last_unstale":"2024-04-25T13:24:13.055016+0000","last_undegraded":"2024-04-25T13:24:13.055016+0000","last_fullsized":"2024-04-25T13:24:13.055016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.408529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877331+0000","last_change":"2024-04-25T13:24:10.861742+0000","last_active":"2024-04-25T13:24:12.877331+0000","last_peered":"2024-04-25T13:24:12.877331+0000","last_clean":"2024-04-25T13:24:12.877331+0000","last_became_active":"2024-04-25T13:24:10.861302+0000","last_became_peered":"2024-04-25T13:24:10.861302+0000","last_unstale":"2024-04-25T13:24:12.877331+0000","last_undegraded":"2024-04-25T13:24:12.877331+0000","last_fullsized":"2024-04-25T13:24:12.877331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.470804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032284+0000","last_change":"2024-04-25T13:24:11.873872+0000","last_active":"2024-04-25T13:24:13.032284+0000","last_peered":"2024-04-25T13:24:13.032284+0000","last_clean":"2024-04-25T13:24:13.032284+0000","last_became_active":"2024-04-25T13:24:11.873341+0000","last_became_peered":"2024-04-25T13:24:11.873341+0000","last_unstale":"2024-04-25T13:24:13.032284+0000","last_undegraded":"2024-04-25T13:24:13.032284+0000","last_fullsized":"2024-04-25T13:24:13.032284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:46.271742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877258+0000","last_change":"2024-04-25T13:24:10.856554+0000","last_active":"2024-04-25T13:24:12.877258+0000","last_peered":"2024-04-25T13:24:12.877258+0000","last_clean":"2024-04-25T13:24:12.877258+0000","last_became_active":"2024-04-25T13:24:10.856254+0000","last_became_peered":"2024-04-25T13:24:10.856254+0000","last_unstale":"2024-04-25T13:24:12.877258+0000","last_undegraded":"2024-04-25T13:24:12.877258+0000","last_fullsized":"2024-04-25T13:24:12.877258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:15.843985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031546+0000","last_change":"2024-04-25T13:24:11.891970+0000","last_active":"2024-04-25T13:24:13.031546+0000","last_peered":"2024-04-25T13:24:13.031546+0000","last_clean":"2024-04-25T13:24:13.031546+0000","last_became_active":"2024-04-25T13:24:11.891537+0000","last_became_peered":"2024-04-25T13:24:11.891537+0000","last_unstale":"2024-04-25T13:24:13.031546+0000","last_undegraded":"2024-04-25T13:24:13.031546+0000","last_fullsized":"2024-04-25T13:24:13.031546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.509206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031238+0000","last_change":"2024-04-25T13:24:10.857130+0000","last_active":"2024-04-25T13:24:13.031238+0000","last_peered":"2024-04-25T13:24:13.031238+0000","last_clean":"2024-04-25T13:24:13.031238+0000","last_became_active":"2024-04-25T13:24:10.856747+0000","last_became_peered":"2024-04-25T13:24:10.856747+0000","last_unstale":"2024-04-25T13:24:13.031238+0000","last_undegraded":"2024-04-25T13:24:13.031238+0000","last_fullsized":"2024-04-25T13:24:13.031238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:11.366223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877158+0000","last_change":"2024-04-25T13:24:11.873404+0000","last_active":"2024-04-25T13:24:12.877158+0000","last_peered":"2024-04-25T13:24:12.877158+0000","last_clean":"2024-04-25T13:24:12.877158+0000","last_became_active":"2024-04-25T13:24:11.873027+0000","last_became_peered":"2024-04-25T13:24:11.873027+0000","last_unstale":"2024-04-25T13:24:12.877158+0000","last_undegraded":"2024-04-25T13:24:12.877158+0000","last_fullsized":"2024-04-25T13:24:12.877158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:32.789300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031959+0000","last_change":"2024-04-25T13:24:10.853602+0000","last_active":"2024-04-25T13:24:13.031959+0000","last_peered":"2024-04-25T13:24:13.031959+0000","last_clean":"2024-04-25T13:24:13.031959+0000","last_became_active":"2024-04-25T13:24:10.853237+0000","last_became_peered":"2024-04-25T13:24:10.853237+0000","last_unstale":"2024-04-25T13:24:13.031959+0000","last_undegraded":"2024-04-25T13:24:13.031959+0000","last_fullsized":"2024-04-25T13:24:13.031959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.575835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031318+0000","last_change":"2024-04-25T13:24:11.891757+0000","last_active":"2024-04-25T13:24:13.031318+0000","last_peered":"2024-04-25T13:24:13.031318+0000","last_clean":"2024-04-25T13:24:13.031318+0000","last_became_active":"2024-04-25T13:24:11.891362+0000","last_became_peered":"2024-04-25T13:24:11.891362+0000","last_unstale":"2024-04-25T13:24:13.031318+0000","last_undegraded":"2024-04-25T13:24:13.031318+0000","last_fullsized":"2024-04-25T13:24:13.031318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:44.202825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030789+0000","last_change":"2024-04-25T13:24:10.851042+0000","last_active":"2024-04-25T13:24:13.030789+0000","last_peered":"2024-04-25T13:24:13.030789+0000","last_clean":"2024-04-25T13:24:13.030789+0000","last_became_active":"2024-04-25T13:24:10.850619+0000","last_became_peered":"2024-04-25T13:24:10.850619+0000","last_unstale":"2024-04-25T13:24:13.030789+0000","last_undegraded":"2024-04-25T13:24:13.030789+0000","last_fullsized":"2024-04-25T13:24:13.030789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:23.577971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876876+0000","last_change":"2024-04-25T13:24:11.880170+0000","last_active":"2024-04-25T13:24:12.876876+0000","last_peered":"2024-04-25T13:24:12.876876+0000","last_clean":"2024-04-25T13:24:12.876876+0000","last_became_active":"2024-04-25T13:24:11.879802+0000","last_became_peered":"2024-04-25T13:24:11.879802+0000","last_unstale":"2024-04-25T13:24:12.876876+0000","last_undegraded":"2024-04-25T13:24:12.876876+0000","last_fullsized":"2024-04-25T13:24:12.876876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:25.231071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030717+0000","last_change":"2024-04-25T13:24:10.852667+0000","last_active":"2024-04-25T13:24:13.030717+0000","last_peered":"2024-04-25T13:24:13.030717+0000","last_clean":"2024-04-25T13:24:13.030717+0000","last_became_active":"2024-04-25T13:24:10.852222+0000","last_became_peered":"2024-04-25T13:24:10.852222+0000","last_unstale":"2024-04-25T13:24:13.030717+0000","last_undegraded":"2024-04-25T13:24:13.030717+0000","last_fullsized":"2024-04-25T13:24:13.030717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:06.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869303+0000","last_change":"2024-04-25T13:24:11.867984+0000","last_active":"2024-04-25T13:24:12.869303+0000","last_peered":"2024-04-25T13:24:12.869303+0000","last_clean":"2024-04-25T13:24:12.869303+0000","last_became_active":"2024-04-25T13:24:11.867517+0000","last_became_peered":"2024-04-25T13:24:11.867517+0000","last_unstale":"2024-04-25T13:24:12.869303+0000","last_undegraded":"2024-04-25T13:24:12.869303+0000","last_fullsized":"2024-04-25T13:24:12.869303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.540573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032434+0000","last_change":"2024-04-25T13:24:10.855925+0000","last_active":"2024-04-25T13:24:13.032434+0000","last_peered":"2024-04-25T13:24:13.032434+0000","last_clean":"2024-04-25T13:24:13.032434+0000","last_became_active":"2024-04-25T13:24:10.855603+0000","last_became_peered":"2024-04-25T13:24:10.855603+0000","last_unstale":"2024-04-25T13:24:13.032434+0000","last_undegraded":"2024-04-25T13:24:13.032434+0000","last_fullsized":"2024-04-25T13:24:13.032434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:26.796979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031354+0000","last_change":"2024-04-25T13:24:11.873683+0000","last_active":"2024-04-25T13:24:13.031354+0000","last_peered":"2024-04-25T13:24:13.031354+0000","last_clean":"2024-04-25T13:24:13.031354+0000","last_became_active":"2024-04-25T13:24:11.873307+0000","last_became_peered":"2024-04-25T13:24:11.873307+0000","last_unstale":"2024-04-25T13:24:13.031354+0000","last_undegraded":"2024-04-25T13:24:13.031354+0000","last_fullsized":"2024-04-25T13:24:13.031354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:44.305349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872528+0000","last_change":"2024-04-25T13:24:10.857124+0000","last_active":"2024-04-25T13:24:12.872528+0000","last_peered":"2024-04-25T13:24:12.872528+0000","last_clean":"2024-04-25T13:24:12.872528+0000","last_became_active":"2024-04-25T13:24:10.856396+0000","last_became_peered":"2024-04-25T13:24:10.856396+0000","last_unstale":"2024-04-25T13:24:12.872528+0000","last_undegraded":"2024-04-25T13:24:12.872528+0000","last_fullsized":"2024-04-25T13:24:12.872528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:58.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.055015+0000","last_change":"2024-04-25T13:24:11.882305+0000","last_active":"2024-04-25T13:24:13.055015+0000","last_peered":"2024-04-25T13:24:13.055015+0000","last_clean":"2024-04-25T13:24:13.055015+0000","last_became_active":"2024-04-25T13:24:11.881976+0000","last_became_peered":"2024-04-25T13:24:11.881976+0000","last_unstale":"2024-04-25T13:24:13.055015+0000","last_undegraded":"2024-04-25T13:24:13.055015+0000","last_fullsized":"2024-04-25T13:24:13.055015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:46.254871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031520+0000","last_change":"2024-04-25T13:24:10.864302+0000","last_active":"2024-04-25T13:24:13.031520+0000","last_peered":"2024-04-25T13:24:13.031520+0000","last_clean":"2024-04-25T13:24:13.031520+0000","last_became_active":"2024-04-25T13:24:10.863955+0000","last_became_peered":"2024-04-25T13:24:10.863955+0000","last_unstale":"2024-04-25T13:24:13.031520+0000","last_undegraded":"2024-04-25T13:24:13.031520+0000","last_fullsized":"2024-04-25T13:24:13.031520+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:35.394778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030883+0000","last_change":"2024-04-25T13:24:11.876293+0000","last_active":"2024-04-25T13:24:13.030883+0000","last_peered":"2024-04-25T13:24:13.030883+0000","last_clean":"2024-04-25T13:24:13.030883+0000","last_became_active":"2024-04-25T13:24:11.875927+0000","last_became_peered":"2024-04-25T13:24:11.875927+0000","last_unstale":"2024-04-25T13:24:13.030883+0000","last_undegraded":"2024-04-25T13:24:13.030883+0000","last_fullsized":"2024-04-25T13:24:13.030883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:47.884328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:14.892043+0000","last_change":"2024-04-25T13:24:10.857046+0000","last_active":"2024-04-25T13:24:14.892043+0000","last_peered":"2024-04-25T13:24:14.892043+0000","last_clean":"2024-04-25T13:24:14.892043+0000","last_became_active":"2024-04-25T13:24:10.856724+0000","last_became_peered":"2024-04-25T13:24:10.856724+0000","last_unstale":"2024-04-25T13:24:14.892043+0000","last_undegraded":"2024-04-25T13:24:14.892043+0000","last_fullsized":"2024-04-25T13:24:14.892043+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:55.913961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877257+0000","last_change":"2024-04-25T13:24:11.886845+0000","last_active":"2024-04-25T13:24:12.877257+0000","last_peered":"2024-04-25T13:24:12.877257+0000","last_clean":"2024-04-25T13:24:12.877257+0000","last_became_active":"2024-04-25T13:24:11.885457+0000","last_became_peered":"2024-04-25T13:24:11.885457+0000","last_unstale":"2024-04-25T13:24:12.877257+0000","last_undegraded":"2024-04-25T13:24:12.877257+0000","last_fullsized":"2024-04-25T13:24:12.877257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.134026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870305+0000","last_change":"2024-04-25T13:24:10.865292+0000","last_active":"2024-04-25T13:24:12.870305+0000","last_peered":"2024-04-25T13:24:12.870305+0000","last_clean":"2024-04-25T13:24:12.870305+0000","last_became_active":"2024-04-25T13:24:10.864940+0000","last_became_peered":"2024-04-25T13:24:10.864940+0000","last_unstale":"2024-04-25T13:24:12.870305+0000","last_undegraded":"2024-04-25T13:24:12.870305+0000","last_fullsized":"2024-04-25T13:24:12.870305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:54.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030956+0000","last_change":"2024-04-25T13:24:11.885158+0000","last_active":"2024-04-25T13:24:13.030956+0000","last_peered":"2024-04-25T13:24:13.030956+0000","last_clean":"2024-04-25T13:24:13.030956+0000","last_became_active":"2024-04-25T13:24:11.884367+0000","last_became_peered":"2024-04-25T13:24:11.884367+0000","last_unstale":"2024-04-25T13:24:13.030956+0000","last_undegraded":"2024-04-25T13:24:13.030956+0000","last_fullsized":"2024-04-25T13:24:13.030956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:48.244922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031218+0000","last_change":"2024-04-25T13:24:10.854269+0000","last_active":"2024-04-25T13:24:13.031218+0000","last_peered":"2024-04-25T13:24:13.031218+0000","last_clean":"2024-04-25T13:24:13.031218+0000","last_became_active":"2024-04-25T13:24:10.853968+0000","last_became_peered":"2024-04-25T13:24:10.853968+0000","last_unstale":"2024-04-25T13:24:13.031218+0000","last_undegraded":"2024-04-25T13:24:13.031218+0000","last_fullsized":"2024-04-25T13:24:13.031218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:42.771743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872675+0000","last_change":"2024-04-25T13:24:11.887468+0000","last_active":"2024-04-25T13:24:12.872675+0000","last_peered":"2024-04-25T13:24:12.872675+0000","last_clean":"2024-04-25T13:24:12.872675+0000","last_became_active":"2024-04-25T13:24:11.887097+0000","last_became_peered":"2024-04-25T13:24:11.887097+0000","last_unstale":"2024-04-25T13:24:12.872675+0000","last_undegraded":"2024-04-25T13:24:12.872675+0000","last_fullsized":"2024-04-25T13:24:12.872675+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:23.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030997+0000","last_change":"2024-04-25T13:24:10.854484+0000","last_active":"2024-04-25T13:24:13.030997+0000","last_peered":"2024-04-25T13:24:13.030997+0000","last_clean":"2024-04-25T13:24:13.030997+0000","last_became_active":"2024-04-25T13:24:10.854152+0000","last_became_peered":"2024-04-25T13:24:10.854152+0000","last_unstale":"2024-04-25T13:24:13.030997+0000","last_undegraded":"2024-04-25T13:24:13.030997+0000","last_fullsized":"2024-04-25T13:24:13.030997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:25.940414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872170+0000","last_change":"2024-04-25T13:24:11.880661+0000","last_active":"2024-04-25T13:24:12.872170+0000","last_peered":"2024-04-25T13:24:12.872170+0000","last_clean":"2024-04-25T13:24:12.872170+0000","last_became_active":"2024-04-25T13:24:11.880099+0000","last_became_peered":"2024-04-25T13:24:11.880099+0000","last_unstale":"2024-04-25T13:24:12.872170+0000","last_undegraded":"2024-04-25T13:24:12.872170+0000","last_fullsized":"2024-04-25T13:24:12.872170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876763+0000","last_change":"2024-04-25T13:24:10.863356+0000","last_active":"2024-04-25T13:24:12.876763+0000","last_peered":"2024-04-25T13:24:12.876763+0000","last_clean":"2024-04-25T13:24:12.876763+0000","last_became_active":"2024-04-25T13:24:10.863024+0000","last_became_peered":"2024-04-25T13:24:10.863024+0000","last_unstale":"2024-04-25T13:24:12.876763+0000","last_undegraded":"2024-04-25T13:24:12.876763+0000","last_fullsized":"2024-04-25T13:24:12.876763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:11.520794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030845+0000","last_change":"2024-04-25T13:24:11.891787+0000","last_active":"2024-04-25T13:24:13.030845+0000","last_peered":"2024-04-25T13:24:13.030845+0000","last_clean":"2024-04-25T13:24:13.030845+0000","last_became_active":"2024-04-25T13:24:11.891466+0000","last_became_peered":"2024-04-25T13:24:11.891466+0000","last_unstale":"2024-04-25T13:24:13.030845+0000","last_undegraded":"2024-04-25T13:24:13.030845+0000","last_fullsized":"2024-04-25T13:24:13.030845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:11.200558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877294+0000","last_change":"2024-04-25T13:24:10.852320+0000","last_active":"2024-04-25T13:24:12.877294+0000","last_peered":"2024-04-25T13:24:12.877294+0000","last_clean":"2024-04-25T13:24:12.877294+0000","last_became_active":"2024-04-25T13:24:10.852007+0000","last_became_peered":"2024-04-25T13:24:10.852007+0000","last_unstale":"2024-04-25T13:24:12.877294+0000","last_undegraded":"2024-04-25T13:24:12.877294+0000","last_fullsized":"2024-04-25T13:24:12.877294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.582461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031489+0000","last_change":"2024-04-25T13:24:11.875174+0000","last_active":"2024-04-25T13:24:13.031489+0000","last_peered":"2024-04-25T13:24:13.031489+0000","last_clean":"2024-04-25T13:24:13.031489+0000","last_became_active":"2024-04-25T13:24:11.874813+0000","last_became_peered":"2024-04-25T13:24:11.874813+0000","last_unstale":"2024-04-25T13:24:13.031489+0000","last_undegraded":"2024-04-25T13:24:13.031489+0000","last_fullsized":"2024-04-25T13:24:13.031489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:22.737412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869343+0000","last_change":"2024-04-25T13:24:10.857715+0000","last_active":"2024-04-25T13:24:12.869343+0000","last_peered":"2024-04-25T13:24:12.869343+0000","last_clean":"2024-04-25T13:24:12.869343+0000","last_became_active":"2024-04-25T13:24:10.857398+0000","last_became_peered":"2024-04-25T13:24:10.857398+0000","last_unstale":"2024-04-25T13:24:12.869343+0000","last_undegraded":"2024-04-25T13:24:12.869343+0000","last_fullsized":"2024-04-25T13:24:12.869343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:58.189652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032597+0000","last_change":"2024-04-25T13:24:11.875065+0000","last_active":"2024-04-25T13:24:13.032597+0000","last_peered":"2024-04-25T13:24:13.032597+0000","last_clean":"2024-04-25T13:24:13.032597+0000","last_became_active":"2024-04-25T13:24:11.874433+0000","last_became_peered":"2024-04-25T13:24:11.874433+0000","last_unstale":"2024-04-25T13:24:13.032597+0000","last_undegraded":"2024-04-25T13:24:13.032597+0000","last_fullsized":"2024-04-25T13:24:13.032597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:07.430007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872227+0000","last_change":"2024-04-25T13:24:10.850250+0000","last_active":"2024-04-25T13:24:12.872227+0000","last_peered":"2024-04-25T13:24:12.872227+0000","last_clean":"2024-04-25T13:24:12.872227+0000","last_became_active":"2024-04-25T13:24:10.849773+0000","last_became_peered":"2024-04-25T13:24:10.849773+0000","last_unstale":"2024-04-25T13:24:12.872227+0000","last_undegraded":"2024-04-25T13:24:12.872227+0000","last_fullsized":"2024-04-25T13:24:12.872227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:59.609421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032615+0000","last_change":"2024-04-25T13:24:11.878625+0000","last_active":"2024-04-25T13:24:13.032615+0000","last_peered":"2024-04-25T13:24:13.032615+0000","last_clean":"2024-04-25T13:24:13.032615+0000","last_became_active":"2024-04-25T13:24:11.878311+0000","last_became_peered":"2024-04-25T13:24:11.878311+0000","last_unstale":"2024-04-25T13:24:13.032615+0000","last_undegraded":"2024-04-25T13:24:13.032615+0000","last_fullsized":"2024-04-25T13:24:13.032615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.363441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031060+0000","last_change":"2024-04-25T13:24:11.887874+0000","last_active":"2024-04-25T13:24:13.031060+0000","last_peered":"2024-04-25T13:24:13.031060+0000","last_clean":"2024-04-25T13:24:13.031060+0000","last_became_active":"2024-04-25T13:24:11.887558+0000","last_became_peered":"2024-04-25T13:24:11.887558+0000","last_unstale":"2024-04-25T13:24:13.031060+0000","last_undegraded":"2024-04-25T13:24:13.031060+0000","last_fullsized":"2024-04-25T13:24:13.031060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:49.861447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030655+0000","last_change":"2024-04-25T13:24:10.859643+0000","last_active":"2024-04-25T13:24:13.030655+0000","last_peered":"2024-04-25T13:24:13.030655+0000","last_clean":"2024-04-25T13:24:13.030655+0000","last_became_active":"2024-04-25T13:24:10.859312+0000","last_became_peered":"2024-04-25T13:24:10.859312+0000","last_unstale":"2024-04-25T13:24:13.030655+0000","last_undegraded":"2024-04-25T13:24:13.030655+0000","last_fullsized":"2024-04-25T13:24:13.030655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.513203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877036+0000","last_change":"2024-04-25T13:24:11.883821+0000","last_active":"2024-04-25T13:24:12.877036+0000","last_peered":"2024-04-25T13:24:12.877036+0000","last_clean":"2024-04-25T13:24:12.877036+0000","last_became_active":"2024-04-25T13:24:11.883063+0000","last_became_peered":"2024-04-25T13:24:11.883063+0000","last_unstale":"2024-04-25T13:24:12.877036+0000","last_undegraded":"2024-04-25T13:24:12.877036+0000","last_fullsized":"2024-04-25T13:24:12.877036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.845694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031477+0000","last_change":"2024-04-25T13:24:10.851525+0000","last_active":"2024-04-25T13:24:13.031477+0000","last_peered":"2024-04-25T13:24:13.031477+0000","last_clean":"2024-04-25T13:24:13.031477+0000","last_became_active":"2024-04-25T13:24:10.851171+0000","last_became_peered":"2024-04-25T13:24:10.851171+0000","last_unstale":"2024-04-25T13:24:13.031477+0000","last_undegraded":"2024-04-25T13:24:13.031477+0000","last_fullsized":"2024-04-25T13:24:13.031477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872078+0000","last_change":"2024-04-25T13:22:58.125848+0000","last_active":"2024-04-25T13:24:12.872078+0000","last_peered":"2024-04-25T13:24:12.872078+0000","last_clean":"2024-04-25T13:24:12.872078+0000","last_became_active":"2024-04-25T13:22:58.125374+0000","last_became_peered":"2024-04-25T13:22:58.125374+0000","last_unstale":"2024-04-25T13:24:12.872078+0000","last_undegraded":"2024-04-25T13:24:12.872078+0000","last_fullsized":"2024-04-25T13:24:12.872078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_clean_scrub_stamp":"2024-04-25T13:19:36.251667+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.304605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030780+0000","last_change":"2024-04-25T13:24:11.883648+0000","last_active":"2024-04-25T13:24:13.030780+0000","last_peered":"2024-04-25T13:24:13.030780+0000","last_clean":"2024-04-25T13:24:13.030780+0000","last_became_active":"2024-04-25T13:24:11.879538+0000","last_became_peered":"2024-04-25T13:24:11.879538+0000","last_unstale":"2024-04-25T13:24:13.030780+0000","last_undegraded":"2024-04-25T13:24:13.030780+0000","last_fullsized":"2024-04-25T13:24:13.030780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872254+0000","last_change":"2024-04-25T13:24:10.851697+0000","last_active":"2024-04-25T13:24:12.872254+0000","last_peered":"2024-04-25T13:24:12.872254+0000","last_clean":"2024-04-25T13:24:12.872254+0000","last_became_active":"2024-04-25T13:24:10.851379+0000","last_became_peered":"2024-04-25T13:24:10.851379+0000","last_unstale":"2024-04-25T13:24:12.872254+0000","last_undegraded":"2024-04-25T13:24:12.872254+0000","last_fullsized":"2024-04-25T13:24:12.872254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:13.104612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877120+0000","last_change":"2024-04-25T13:24:10.864221+0000","last_active":"2024-04-25T13:24:12.877120+0000","last_peered":"2024-04-25T13:24:12.877120+0000","last_clean":"2024-04-25T13:24:12.877120+0000","last_became_active":"2024-04-25T13:24:10.863892+0000","last_became_peered":"2024-04-25T13:24:10.863892+0000","last_unstale":"2024-04-25T13:24:12.877120+0000","last_undegraded":"2024-04-25T13:24:12.877120+0000","last_fullsized":"2024-04-25T13:24:12.877120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:51.229865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031120+0000","last_change":"2024-04-25T13:24:11.878257+0000","last_active":"2024-04-25T13:24:13.031120+0000","last_peered":"2024-04-25T13:24:13.031120+0000","last_clean":"2024-04-25T13:24:13.031120+0000","last_became_active":"2024-04-25T13:24:11.877926+0000","last_became_peered":"2024-04-25T13:24:11.877926+0000","last_unstale":"2024-04-25T13:24:13.031120+0000","last_undegraded":"2024-04-25T13:24:13.031120+0000","last_fullsized":"2024-04-25T13:24:13.031120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:32.873011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869522+0000","last_change":"2024-04-25T13:24:10.860867+0000","last_active":"2024-04-25T13:24:12.869522+0000","last_peered":"2024-04-25T13:24:12.869522+0000","last_clean":"2024-04-25T13:24:12.869522+0000","last_became_active":"2024-04-25T13:24:10.860566+0000","last_became_peered":"2024-04-25T13:24:10.860566+0000","last_unstale":"2024-04-25T13:24:12.869522+0000","last_undegraded":"2024-04-25T13:24:12.869522+0000","last_fullsized":"2024-04-25T13:24:12.869522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.340184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877157+0000","last_change":"2024-04-25T13:24:11.884573+0000","last_active":"2024-04-25T13:24:12.877157+0000","last_peered":"2024-04-25T13:24:12.877157+0000","last_clean":"2024-04-25T13:24:12.877157+0000","last_became_active":"2024-04-25T13:24:11.884245+0000","last_became_peered":"2024-04-25T13:24:11.884245+0000","last_unstale":"2024-04-25T13:24:12.877157+0000","last_undegraded":"2024-04-25T13:24:12.877157+0000","last_fullsized":"2024-04-25T13:24:12.877157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:12.555396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031355+0000","last_change":"2024-04-25T13:24:10.854441+0000","last_active":"2024-04-25T13:24:13.031355+0000","last_peered":"2024-04-25T13:24:13.031355+0000","last_clean":"2024-04-25T13:24:13.031355+0000","last_became_active":"2024-04-25T13:24:10.854086+0000","last_became_peered":"2024-04-25T13:24:10.854086+0000","last_unstale":"2024-04-25T13:24:13.031355+0000","last_undegraded":"2024-04-25T13:24:13.031355+0000","last_fullsized":"2024-04-25T13:24:13.031355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:17.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054861+0000","last_change":"2024-04-25T13:24:11.880494+0000","last_active":"2024-04-25T13:24:13.054861+0000","last_peered":"2024-04-25T13:24:13.054861+0000","last_clean":"2024-04-25T13:24:13.054861+0000","last_became_active":"2024-04-25T13:24:11.880074+0000","last_became_peered":"2024-04-25T13:24:11.880074+0000","last_unstale":"2024-04-25T13:24:13.054861+0000","last_undegraded":"2024-04-25T13:24:13.054861+0000","last_fullsized":"2024-04-25T13:24:13.054861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:50.993165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032176+0000","last_change":"2024-04-25T13:24:10.863065+0000","last_active":"2024-04-25T13:24:13.032176+0000","last_peered":"2024-04-25T13:24:13.032176+0000","last_clean":"2024-04-25T13:24:13.032176+0000","last_became_active":"2024-04-25T13:24:10.862793+0000","last_became_peered":"2024-04-25T13:24:10.862793+0000","last_unstale":"2024-04-25T13:24:13.032176+0000","last_undegraded":"2024-04-25T13:24:13.032176+0000","last_fullsized":"2024-04-25T13:24:13.032176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:03.719745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876929+0000","last_change":"2024-04-25T13:24:11.878770+0000","last_active":"2024-04-25T13:24:12.876929+0000","last_peered":"2024-04-25T13:24:12.876929+0000","last_clean":"2024-04-25T13:24:12.876929+0000","last_became_active":"2024-04-25T13:24:11.878424+0000","last_became_peered":"2024-04-25T13:24:11.878424+0000","last_unstale":"2024-04-25T13:24:12.876929+0000","last_undegraded":"2024-04-25T13:24:12.876929+0000","last_fullsized":"2024-04-25T13:24:12.876929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:25.016200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876720+0000","last_change":"2024-04-25T13:24:10.857246+0000","last_active":"2024-04-25T13:24:12.876720+0000","last_peered":"2024-04-25T13:24:12.876720+0000","last_clean":"2024-04-25T13:24:12.876720+0000","last_became_active":"2024-04-25T13:24:10.856922+0000","last_became_peered":"2024-04-25T13:24:10.856922+0000","last_unstale":"2024-04-25T13:24:12.876720+0000","last_undegraded":"2024-04-25T13:24:12.876720+0000","last_fullsized":"2024-04-25T13:24:12.876720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:08.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030750+0000","last_change":"2024-04-25T13:24:11.873225+0000","last_active":"2024-04-25T13:24:13.030750+0000","last_peered":"2024-04-25T13:24:13.030750+0000","last_clean":"2024-04-25T13:24:13.030750+0000","last_became_active":"2024-04-25T13:24:11.872855+0000","last_became_peered":"2024-04-25T13:24:11.872855+0000","last_unstale":"2024-04-25T13:24:13.030750+0000","last_undegraded":"2024-04-25T13:24:13.030750+0000","last_fullsized":"2024-04-25T13:24:13.030750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.523621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870070+0000","last_change":"2024-04-25T13:24:10.863165+0000","last_active":"2024-04-25T13:24:12.870070+0000","last_peered":"2024-04-25T13:24:12.870070+0000","last_clean":"2024-04-25T13:24:12.870070+0000","last_became_active":"2024-04-25T13:24:10.862596+0000","last_became_peered":"2024-04-25T13:24:10.862596+0000","last_unstale":"2024-04-25T13:24:12.870070+0000","last_undegraded":"2024-04-25T13:24:12.870070+0000","last_fullsized":"2024-04-25T13:24:12.870070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:51.414158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877091+0000","last_change":"2024-04-25T13:24:11.885852+0000","last_active":"2024-04-25T13:24:12.877091+0000","last_peered":"2024-04-25T13:24:12.877091+0000","last_clean":"2024-04-25T13:24:12.877091+0000","last_became_active":"2024-04-25T13:24:11.885508+0000","last_became_peered":"2024-04-25T13:24:11.885508+0000","last_unstale":"2024-04-25T13:24:12.877091+0000","last_undegraded":"2024-04-25T13:24:12.877091+0000","last_fullsized":"2024-04-25T13:24:12.877091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.575351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877007+0000","last_change":"2024-04-25T13:24:10.861701+0000","last_active":"2024-04-25T13:24:12.877007+0000","last_peered":"2024-04-25T13:24:12.877007+0000","last_clean":"2024-04-25T13:24:12.877007+0000","last_became_active":"2024-04-25T13:24:10.861356+0000","last_became_peered":"2024-04-25T13:24:10.861356+0000","last_unstale":"2024-04-25T13:24:12.877007+0000","last_undegraded":"2024-04-25T13:24:12.877007+0000","last_fullsized":"2024-04-25T13:24:12.877007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:24.300835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872496+0000","last_change":"2024-04-25T13:24:11.883977+0000","last_active":"2024-04-25T13:24:12.872496+0000","last_peered":"2024-04-25T13:24:12.872496+0000","last_clean":"2024-04-25T13:24:12.872496+0000","last_became_active":"2024-04-25T13:24:11.883401+0000","last_became_peered":"2024-04-25T13:24:11.883401+0000","last_unstale":"2024-04-25T13:24:12.872496+0000","last_undegraded":"2024-04-25T13:24:12.872496+0000","last_fullsized":"2024-04-25T13:24:12.872496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:04.291865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870384+0000","last_change":"2024-04-25T13:24:10.858303+0000","last_active":"2024-04-25T13:24:12.870384+0000","last_peered":"2024-04-25T13:24:12.870384+0000","last_clean":"2024-04-25T13:24:12.870384+0000","last_became_active":"2024-04-25T13:24:10.857956+0000","last_became_peered":"2024-04-25T13:24:10.857956+0000","last_unstale":"2024-04-25T13:24:12.870384+0000","last_undegraded":"2024-04-25T13:24:12.870384+0000","last_fullsized":"2024-04-25T13:24:12.870384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.753759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872720+0000","last_change":"2024-04-25T13:24:11.882367+0000","last_active":"2024-04-25T13:24:12.872720+0000","last_peered":"2024-04-25T13:24:12.872720+0000","last_clean":"2024-04-25T13:24:12.872720+0000","last_became_active":"2024-04-25T13:24:11.880495+0000","last_became_peered":"2024-04-25T13:24:11.880495+0000","last_unstale":"2024-04-25T13:24:12.872720+0000","last_undegraded":"2024-04-25T13:24:12.872720+0000","last_fullsized":"2024-04-25T13:24:12.872720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.031326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031381+0000","last_change":"2024-04-25T13:24:10.854984+0000","last_active":"2024-04-25T13:24:13.031381+0000","last_peered":"2024-04-25T13:24:13.031381+0000","last_clean":"2024-04-25T13:24:13.031381+0000","last_became_active":"2024-04-25T13:24:10.854691+0000","last_became_peered":"2024-04-25T13:24:10.854691+0000","last_unstale":"2024-04-25T13:24:13.031381+0000","last_undegraded":"2024-04-25T13:24:13.031381+0000","last_fullsized":"2024-04-25T13:24:13.031381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:55.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872420+0000","last_change":"2024-04-25T13:24:11.881329+0000","last_active":"2024-04-25T13:24:12.872420+0000","last_peered":"2024-04-25T13:24:12.872420+0000","last_clean":"2024-04-25T13:24:12.872420+0000","last_became_active":"2024-04-25T13:24:11.880958+0000","last_became_peered":"2024-04-25T13:24:11.880958+0000","last_unstale":"2024-04-25T13:24:12.872420+0000","last_undegraded":"2024-04-25T13:24:12.872420+0000","last_fullsized":"2024-04-25T13:24:12.872420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:54.720129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030876+0000","last_change":"2024-04-25T13:24:10.861501+0000","last_active":"2024-04-25T13:24:13.030876+0000","last_peered":"2024-04-25T13:24:13.030876+0000","last_clean":"2024-04-25T13:24:13.030876+0000","last_became_active":"2024-04-25T13:24:10.861155+0000","last_became_peered":"2024-04-25T13:24:10.861155+0000","last_unstale":"2024-04-25T13:24:13.030876+0000","last_undegraded":"2024-04-25T13:24:13.030876+0000","last_fullsized":"2024-04-25T13:24:13.030876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:17.306214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872835+0000","last_change":"2024-04-25T13:24:11.879779+0000","last_active":"2024-04-25T13:24:12.872835+0000","last_peered":"2024-04-25T13:24:12.872835+0000","last_clean":"2024-04-25T13:24:12.872835+0000","last_became_active":"2024-04-25T13:24:11.879285+0000","last_became_peered":"2024-04-25T13:24:11.879285+0000","last_unstale":"2024-04-25T13:24:12.872835+0000","last_undegraded":"2024-04-25T13:24:12.872835+0000","last_fullsized":"2024-04-25T13:24:12.872835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.799094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031003+0000","last_change":"2024-04-25T13:24:11.871059+0000","last_active":"2024-04-25T13:24:13.031003+0000","last_peered":"2024-04-25T13:24:13.031003+0000","last_clean":"2024-04-25T13:24:13.031003+0000","last_became_active":"2024-04-25T13:24:11.869530+0000","last_became_peered":"2024-04-25T13:24:11.869530+0000","last_unstale":"2024-04-25T13:24:13.031003+0000","last_undegraded":"2024-04-25T13:24:13.031003+0000","last_fullsized":"2024-04-25T13:24:13.031003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.095146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870339+0000","last_change":"2024-04-25T13:24:11.876162+0000","last_active":"2024-04-25T13:24:12.870339+0000","last_peered":"2024-04-25T13:24:12.870339+0000","last_clean":"2024-04-25T13:24:12.870339+0000","last_became_active":"2024-04-25T13:24:11.875747+0000","last_became_peered":"2024-04-25T13:24:11.875747+0000","last_unstale":"2024-04-25T13:24:12.870339+0000","last_undegraded":"2024-04-25T13:24:12.870339+0000","last_fullsized":"2024-04-25T13:24:12.870339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:07.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870214+0000","last_change":"2024-04-25T13:24:11.887477+0000","last_active":"2024-04-25T13:24:12.870214+0000","last_peered":"2024-04-25T13:24:12.870214+0000","last_clean":"2024-04-25T13:24:12.870214+0000","last_became_active":"2024-04-25T13:24:11.886951+0000","last_became_peered":"2024-04-25T13:24:11.886951+0000","last_unstale":"2024-04-25T13:24:12.870214+0000","last_undegraded":"2024-04-25T13:24:12.870214+0000","last_fullsized":"2024-04-25T13:24:12.870214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:28.865817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054680+0000","last_change":"2024-04-25T13:24:11.882545+0000","last_active":"2024-04-25T13:24:13.054680+0000","last_peered":"2024-04-25T13:24:13.054680+0000","last_clean":"2024-04-25T13:24:13.054680+0000","last_became_active":"2024-04-25T13:24:11.881461+0000","last_became_peered":"2024-04-25T13:24:11.881461+0000","last_unstale":"2024-04-25T13:24:13.054680+0000","last_undegraded":"2024-04-25T13:24:13.054680+0000","last_fullsized":"2024-04-25T13:24:13.054680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:26.823813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031085+0000","last_change":"2024-04-25T13:24:11.876929+0000","last_active":"2024-04-25T13:24:13.031085+0000","last_peered":"2024-04-25T13:24:13.031085+0000","last_clean":"2024-04-25T13:24:13.031085+0000","last_became_active":"2024-04-25T13:24:11.876578+0000","last_became_peered":"2024-04-25T13:24:11.876578+0000","last_unstale":"2024-04-25T13:24:13.031085+0000","last_undegraded":"2024-04-25T13:24:13.031085+0000","last_fullsized":"2024-04-25T13:24:13.031085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.621282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031200+0000","last_change":"2024-04-25T13:24:11.869478+0000","last_active":"2024-04-25T13:24:13.031200+0000","last_peered":"2024-04-25T13:24:13.031200+0000","last_clean":"2024-04-25T13:24:13.031200+0000","last_became_active":"2024-04-25T13:24:11.869104+0000","last_became_peered":"2024-04-25T13:24:11.869104+0000","last_unstale":"2024-04-25T13:24:13.031200+0000","last_undegraded":"2024-04-25T13:24:13.031200+0000","last_fullsized":"2024-04-25T13:24:13.031200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:46.865868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870664+0000","last_change":"2024-04-25T13:24:11.874008+0000","last_active":"2024-04-25T13:24:12.870664+0000","last_peered":"2024-04-25T13:24:12.870664+0000","last_clean":"2024-04-25T13:24:12.870664+0000","last_became_active":"2024-04-25T13:24:11.873576+0000","last_became_peered":"2024-04-25T13:24:11.873576+0000","last_unstale":"2024-04-25T13:24:12.870664+0000","last_undegraded":"2024-04-25T13:24:12.870664+0000","last_fullsized":"2024-04-25T13:24:12.870664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:14.395329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877099+0000","last_change":"2024-04-25T13:24:11.880749+0000","last_active":"2024-04-25T13:24:12.877099+0000","last_peered":"2024-04-25T13:24:12.877099+0000","last_clean":"2024-04-25T13:24:12.877099+0000","last_became_active":"2024-04-25T13:24:11.880421+0000","last_became_peered":"2024-04-25T13:24:11.880421+0000","last_unstale":"2024-04-25T13:24:12.877099+0000","last_undegraded":"2024-04-25T13:24:12.877099+0000","last_fullsized":"2024-04-25T13:24:12.877099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:44.001354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030844+0000","last_change":"2024-04-25T13:24:11.885922+0000","last_active":"2024-04-25T13:24:13.030844+0000","last_peered":"2024-04-25T13:24:13.030844+0000","last_clean":"2024-04-25T13:24:13.030844+0000","last_became_active":"2024-04-25T13:24:11.885611+0000","last_became_peered":"2024-04-25T13:24:11.885611+0000","last_unstale":"2024-04-25T13:24:13.030844+0000","last_undegraded":"2024-04-25T13:24:13.030844+0000","last_fullsized":"2024-04-25T13:24:13.030844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876857+0000","last_change":"2024-04-25T13:24:11.877912+0000","last_active":"2024-04-25T13:24:12.876857+0000","last_peered":"2024-04-25T13:24:12.876857+0000","last_clean":"2024-04-25T13:24:12.876857+0000","last_became_active":"2024-04-25T13:24:11.877139+0000","last_became_peered":"2024-04-25T13:24:11.877139+0000","last_unstale":"2024-04-25T13:24:12.876857+0000","last_undegraded":"2024-04-25T13:24:12.876857+0000","last_fullsized":"2024-04-25T13:24:12.876857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:45.618556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032263+0000","last_change":"2024-04-25T13:24:11.889267+0000","last_active":"2024-04-25T13:24:13.032263+0000","last_peered":"2024-04-25T13:24:13.032263+0000","last_clean":"2024-04-25T13:24:13.032263+0000","last_became_active":"2024-04-25T13:24:11.888875+0000","last_became_peered":"2024-04-25T13:24:11.888875+0000","last_unstale":"2024-04-25T13:24:13.032263+0000","last_undegraded":"2024-04-25T13:24:13.032263+0000","last_fullsized":"2024-04-25T13:24:13.032263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:41.871075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031125+0000","last_change":"2024-04-25T13:24:11.890664+0000","last_active":"2024-04-25T13:24:13.031125+0000","last_peered":"2024-04-25T13:24:13.031125+0000","last_clean":"2024-04-25T13:24:13.031125+0000","last_became_active":"2024-04-25T13:24:11.890124+0000","last_became_peered":"2024-04-25T13:24:11.890124+0000","last_unstale":"2024-04-25T13:24:13.031125+0000","last_undegraded":"2024-04-25T13:24:13.031125+0000","last_fullsized":"2024-04-25T13:24:13.031125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:55.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031031+0000","last_change":"2024-04-25T13:24:11.876201+0000","last_active":"2024-04-25T13:24:13.031031+0000","last_peered":"2024-04-25T13:24:13.031031+0000","last_clean":"2024-04-25T13:24:13.031031+0000","last_became_active":"2024-04-25T13:24:11.875783+0000","last_became_peered":"2024-04-25T13:24:11.875783+0000","last_unstale":"2024-04-25T13:24:13.031031+0000","last_undegraded":"2024-04-25T13:24:13.031031+0000","last_fullsized":"2024-04-25T13:24:13.031031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.264576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":13830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":456350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1572864,"data_stored":1048524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678040,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":456932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939678,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":456954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234019,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1552384,"data_stored":1047300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":954368,"data_stored":456350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":460368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":456440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":457996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117046,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":456454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":456372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691884,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1560576,"data_stored":1046900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:24:18.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 -- 172.21.15.37:0/726253872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 msgr2=0x7f7388107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 --2- 172.21.15.37:0/726253872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f7388107750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7370009900 tx=0x7f737002a410 comp rx=0 tx=0).stop 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 -- 172.21.15.37:0/726253872 shutdown_connections 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 --2- 172.21.15.37:0/726253872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 0x7f738810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 --2- 172.21.15.37:0/726253872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f7388107750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 --2- 172.21.15.37:0/726253872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f7388104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 -- 172.21.15.37:0/726253872 >> 172.21.15.37:0/726253872 conn(0x7f73880fc460 msgr2=0x7f73880fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:18.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 -- 172.21.15.37:0/726253872 shutdown_connections 2024-04-25T13:24:18.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.847+0000 7f738e4e8640 1 -- 172.21.15.37:0/726253872 wait complete. 2024-04-25T13:24:18.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 Processor -- start 2024-04-25T13:24:18.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 -- start start 2024-04-25T13:24:18.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f73881981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 0x7f738819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f738810dab0 con 0x7f7388105360 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f738810d930 con 0x7f7388107c90 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f738e4e8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f738810dc30 con 0x7f7388102a30 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f73877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f73877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49404/0 (socket says 172.21.15.37:49404) 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f73877fe640 1 -- 172.21.15.37:0/1541160505 learned_addr learned my addr 172.21.15.37:0/1541160505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.848+0000 7f7387fff640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f73881981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f738ca85640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 0x7f738819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 -- 172.21.15.37:0/1541160505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 msgr2=0x7f73881981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f73881981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 -- 172.21.15.37:0/1541160505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 msgr2=0x7f738819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 0x7f738819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:18.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 -- 172.21.15.37:0/1541160505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738819fdd0 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f7387fff640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f73881981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73877fe640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7370006010 tx=0x7f73700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737003f030 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7370003cb0 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73881a0060 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370035070 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.849+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73881a0550 con 0x7f7388105360 2024-04-25T13:24:18.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.850+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73880fdf50 con 0x7f7388105360 2024-04-25T13:24:18.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.851+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7370042030 con 0x7f7388105360 2024-04-25T13:24:18.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.852+0000 7f73857fa640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 0x7f73740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:18.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.852+0000 7f7387fff640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 0x7f73740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:18.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.852+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f73700f0a30 con 0x7f7388105360 2024-04-25T13:24:18.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.853+0000 7f7387fff640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 0x7f73740a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f737800afb0 tx=0x7f737800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:18.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:18.857+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73700bd460 con 0x7f7388105360 2024-04-25T13:24:19.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.045+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7388062390 con 0x7f73740a3c40 2024-04-25T13:24:19.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.052+0000 7f73857fa640 1 -- 172.21.15.37:0/1541160505 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258724 (secure 0 0 0) 0x7f7388062390 con 0x7f73740a3c40 2024-04-25T13:24:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:19.056 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T13:24:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 msgr2=0x7f73740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 0x7f73740a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f737800afb0 tx=0x7f737800a2d0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 msgr2=0x7f73881986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7370006010 tx=0x7f73700042f0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 shutdown_connections 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7388107c90 0x7f738819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73740a3c40 0x7f73740a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7388105360 0x7f73881986f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 --2- 172.21.15.37:0/1541160505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7388102a30 0x7f73881981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 >> 172.21.15.37:0/1541160505 conn(0x7f73880fc460 msgr2=0x7f73881034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 shutdown_connections 2024-04-25T13:24:19.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:19.055+0000 7f738e4e8640 1 -- 172.21.15.37:0/1541160505 wait complete. 2024-04-25T13:24:19.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:18 smithi037 ceph-mon[29204]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:19.065 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:18 smithi037 ceph-mon[29204]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T13:24:19.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:18 smithi037 ceph-mon[29204]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:24:19.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:18 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/831991281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:19.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:18 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2357652895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:19.067 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-04-25T13:24:17.730144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":145,"num_read_kb":127,"num_write":226,"num_write_kb":2952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":176,"ondisk_log_size":176,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":408044,"kb_used_data":13020,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124533780,"statfs":{"total":1151940427776,"available":1151522590720,"internally_reserved":0,"allocated":13332480,"data_stored":7256940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":404272442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.890751"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031512+0000","last_change":"2024-04-25T13:24:11.873737+0000","last_active":"2024-04-25T13:24:13.031512+0000","last_peered":"2024-04-25T13:24:13.031512+0000","last_clean":"2024-04-25T13:24:13.031512+0000","last_became_active":"2024-04-25T13:24:11.872598+0000","last_became_peered":"2024-04-25T13:24:11.872598+0000","last_unstale":"2024-04-25T13:24:13.031512+0000","last_undegraded":"2024-04-25T13:24:13.031512+0000","last_fullsized":"2024-04-25T13:24:13.031512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:46.739869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054856+0000","last_change":"2024-04-25T13:24:11.886834+0000","last_active":"2024-04-25T13:24:13.054856+0000","last_peered":"2024-04-25T13:24:13.054856+0000","last_clean":"2024-04-25T13:24:13.054856+0000","last_became_active":"2024-04-25T13:24:11.886460+0000","last_became_peered":"2024-04-25T13:24:11.886460+0000","last_unstale":"2024-04-25T13:24:13.054856+0000","last_undegraded":"2024-04-25T13:24:13.054856+0000","last_fullsized":"2024-04-25T13:24:13.054856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.103106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054721+0000","last_change":"2024-04-25T13:24:11.877582+0000","last_active":"2024-04-25T13:24:13.054721+0000","last_peered":"2024-04-25T13:24:13.054721+0000","last_clean":"2024-04-25T13:24:13.054721+0000","last_became_active":"2024-04-25T13:24:11.877245+0000","last_became_peered":"2024-04-25T13:24:11.877245+0000","last_unstale":"2024-04-25T13:24:13.054721+0000","last_undegraded":"2024-04-25T13:24:13.054721+0000","last_fullsized":"2024-04-25T13:24:13.054721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:05.065255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031844+0000","last_change":"2024-04-25T13:24:11.873522+0000","last_active":"2024-04-25T13:24:13.031844+0000","last_peered":"2024-04-25T13:24:13.031844+0000","last_clean":"2024-04-25T13:24:13.031844+0000","last_became_active":"2024-04-25T13:24:11.872505+0000","last_became_peered":"2024-04-25T13:24:11.872505+0000","last_unstale":"2024-04-25T13:24:13.031844+0000","last_undegraded":"2024-04-25T13:24:13.031844+0000","last_fullsized":"2024-04-25T13:24:13.031844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:15.968935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876985+0000","last_change":"2024-04-25T13:24:11.873353+0000","last_active":"2024-04-25T13:24:12.876985+0000","last_peered":"2024-04-25T13:24:12.876985+0000","last_clean":"2024-04-25T13:24:12.876985+0000","last_became_active":"2024-04-25T13:24:11.872997+0000","last_became_peered":"2024-04-25T13:24:11.872997+0000","last_unstale":"2024-04-25T13:24:12.876985+0000","last_undegraded":"2024-04-25T13:24:12.876985+0000","last_fullsized":"2024-04-25T13:24:12.876985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:15.205105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032139+0000","last_change":"2024-04-25T13:24:11.875994+0000","last_active":"2024-04-25T13:24:13.032139+0000","last_peered":"2024-04-25T13:24:13.032139+0000","last_clean":"2024-04-25T13:24:13.032139+0000","last_became_active":"2024-04-25T13:24:11.875658+0000","last_became_peered":"2024-04-25T13:24:11.875658+0000","last_unstale":"2024-04-25T13:24:13.032139+0000","last_undegraded":"2024-04-25T13:24:13.032139+0000","last_fullsized":"2024-04-25T13:24:13.032139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:20.959715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872229+0000","last_change":"2024-04-25T13:24:11.879417+0000","last_active":"2024-04-25T13:24:12.872229+0000","last_peered":"2024-04-25T13:24:12.872229+0000","last_clean":"2024-04-25T13:24:12.872229+0000","last_became_active":"2024-04-25T13:24:11.879112+0000","last_became_peered":"2024-04-25T13:24:11.879112+0000","last_unstale":"2024-04-25T13:24:12.872229+0000","last_undegraded":"2024-04-25T13:24:12.872229+0000","last_fullsized":"2024-04-25T13:24:12.872229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:25.250253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876730+0000","last_change":"2024-04-25T13:24:11.883661+0000","last_active":"2024-04-25T13:24:12.876730+0000","last_peered":"2024-04-25T13:24:12.876730+0000","last_clean":"2024-04-25T13:24:12.876730+0000","last_became_active":"2024-04-25T13:24:11.883219+0000","last_became_peered":"2024-04-25T13:24:11.883219+0000","last_unstale":"2024-04-25T13:24:12.876730+0000","last_undegraded":"2024-04-25T13:24:12.876730+0000","last_fullsized":"2024-04-25T13:24:12.876730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:15.731727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031629+0000","last_change":"2024-04-25T13:24:11.880350+0000","last_active":"2024-04-25T13:24:13.031629+0000","last_peered":"2024-04-25T13:24:13.031629+0000","last_clean":"2024-04-25T13:24:13.031629+0000","last_became_active":"2024-04-25T13:24:11.880028+0000","last_became_peered":"2024-04-25T13:24:11.880028+0000","last_unstale":"2024-04-25T13:24:13.031629+0000","last_undegraded":"2024-04-25T13:24:13.031629+0000","last_fullsized":"2024-04-25T13:24:13.031629+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:54.832380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032641+0000","last_change":"2024-04-25T13:24:11.879180+0000","last_active":"2024-04-25T13:24:13.032641+0000","last_peered":"2024-04-25T13:24:13.032641+0000","last_clean":"2024-04-25T13:24:13.032641+0000","last_became_active":"2024-04-25T13:24:11.877612+0000","last_became_peered":"2024-04-25T13:24:11.877612+0000","last_unstale":"2024-04-25T13:24:13.032641+0000","last_undegraded":"2024-04-25T13:24:13.032641+0000","last_fullsized":"2024-04-25T13:24:13.032641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.269957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872729+0000","last_change":"2024-04-25T13:24:11.880768+0000","last_active":"2024-04-25T13:24:12.872729+0000","last_peered":"2024-04-25T13:24:12.872729+0000","last_clean":"2024-04-25T13:24:12.872729+0000","last_became_active":"2024-04-25T13:24:11.879900+0000","last_became_peered":"2024-04-25T13:24:11.879900+0000","last_unstale":"2024-04-25T13:24:12.872729+0000","last_undegraded":"2024-04-25T13:24:12.872729+0000","last_fullsized":"2024-04-25T13:24:12.872729+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:11.941761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031611+0000","last_change":"2024-04-25T13:24:11.873868+0000","last_active":"2024-04-25T13:24:13.031611+0000","last_peered":"2024-04-25T13:24:13.031611+0000","last_clean":"2024-04-25T13:24:13.031611+0000","last_became_active":"2024-04-25T13:24:11.872941+0000","last_became_peered":"2024-04-25T13:24:11.872941+0000","last_unstale":"2024-04-25T13:24:13.031611+0000","last_undegraded":"2024-04-25T13:24:13.031611+0000","last_fullsized":"2024-04-25T13:24:13.031611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:15.580750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031013+0000","last_change":"2024-04-25T13:24:11.888054+0000","last_active":"2024-04-25T13:24:13.031013+0000","last_peered":"2024-04-25T13:24:13.031013+0000","last_clean":"2024-04-25T13:24:13.031013+0000","last_became_active":"2024-04-25T13:24:11.887217+0000","last_became_peered":"2024-04-25T13:24:11.887217+0000","last_unstale":"2024-04-25T13:24:13.031013+0000","last_undegraded":"2024-04-25T13:24:13.031013+0000","last_fullsized":"2024-04-25T13:24:13.031013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:08.058645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054548+0000","last_change":"2024-04-25T13:24:11.878049+0000","last_active":"2024-04-25T13:24:13.054548+0000","last_peered":"2024-04-25T13:24:13.054548+0000","last_clean":"2024-04-25T13:24:13.054548+0000","last_became_active":"2024-04-25T13:24:11.876595+0000","last_became_peered":"2024-04-25T13:24:11.876595+0000","last_unstale":"2024-04-25T13:24:13.054548+0000","last_undegraded":"2024-04-25T13:24:13.054548+0000","last_fullsized":"2024-04-25T13:24:13.054548+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:36.126016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054560+0000","last_change":"2024-04-25T13:24:11.890173+0000","last_active":"2024-04-25T13:24:13.054560+0000","last_peered":"2024-04-25T13:24:13.054560+0000","last_clean":"2024-04-25T13:24:13.054560+0000","last_became_active":"2024-04-25T13:24:11.889865+0000","last_became_peered":"2024-04-25T13:24:11.889865+0000","last_unstale":"2024-04-25T13:24:13.054560+0000","last_undegraded":"2024-04-25T13:24:13.054560+0000","last_fullsized":"2024-04-25T13:24:13.054560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:36.809979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030690+0000","last_change":"2024-04-25T13:24:11.888024+0000","last_active":"2024-04-25T13:24:13.030690+0000","last_peered":"2024-04-25T13:24:13.030690+0000","last_clean":"2024-04-25T13:24:13.030690+0000","last_became_active":"2024-04-25T13:24:11.887497+0000","last_became_peered":"2024-04-25T13:24:11.887497+0000","last_unstale":"2024-04-25T13:24:13.030690+0000","last_undegraded":"2024-04-25T13:24:13.030690+0000","last_fullsized":"2024-04-25T13:24:13.030690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.887282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031702+0000","last_change":"2024-04-25T13:24:11.872532+0000","last_active":"2024-04-25T13:24:13.031702+0000","last_peered":"2024-04-25T13:24:13.031702+0000","last_clean":"2024-04-25T13:24:13.031702+0000","last_became_active":"2024-04-25T13:24:11.871047+0000","last_became_peered":"2024-04-25T13:24:11.871047+0000","last_unstale":"2024-04-25T13:24:13.031702+0000","last_undegraded":"2024-04-25T13:24:13.031702+0000","last_fullsized":"2024-04-25T13:24:13.031702+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:21.818593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869606+0000","last_change":"2024-04-25T13:24:11.869003+0000","last_active":"2024-04-25T13:24:12.869606+0000","last_peered":"2024-04-25T13:24:12.869606+0000","last_clean":"2024-04-25T13:24:12.869606+0000","last_became_active":"2024-04-25T13:24:11.868639+0000","last_became_peered":"2024-04-25T13:24:11.868639+0000","last_unstale":"2024-04-25T13:24:12.869606+0000","last_undegraded":"2024-04-25T13:24:12.869606+0000","last_fullsized":"2024-04-25T13:24:12.869606+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:03.366661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870511+0000","last_change":"2024-04-25T13:24:11.878947+0000","last_active":"2024-04-25T13:24:12.870511+0000","last_peered":"2024-04-25T13:24:12.870511+0000","last_clean":"2024-04-25T13:24:12.870511+0000","last_became_active":"2024-04-25T13:24:11.878344+0000","last_became_peered":"2024-04-25T13:24:11.878344+0000","last_unstale":"2024-04-25T13:24:12.870511+0000","last_undegraded":"2024-04-25T13:24:12.870511+0000","last_fullsized":"2024-04-25T13:24:12.870511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:54.653873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'4","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869560+0000","last_change":"2024-04-25T13:24:10.856180+0000","last_active":"2024-04-25T13:24:12.869560+0000","last_peered":"2024-04-25T13:24:12.869560+0000","last_clean":"2024-04-25T13:24:12.869560+0000","last_became_active":"2024-04-25T13:24:10.855746+0000","last_became_peered":"2024-04-25T13:24:10.855746+0000","last_unstale":"2024-04-25T13:24:12.869560+0000","last_undegraded":"2024-04-25T13:24:12.869560+0000","last_fullsized":"2024-04-25T13:24:12.869560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:58.312713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877118+0000","last_change":"2024-04-25T13:24:11.889286+0000","last_active":"2024-04-25T13:24:12.877118+0000","last_peered":"2024-04-25T13:24:12.877118+0000","last_clean":"2024-04-25T13:24:12.877118+0000","last_became_active":"2024-04-25T13:24:11.888950+0000","last_became_peered":"2024-04-25T13:24:11.888950+0000","last_unstale":"2024-04-25T13:24:12.877118+0000","last_undegraded":"2024-04-25T13:24:12.877118+0000","last_fullsized":"2024-04-25T13:24:12.877118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:31.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877190+0000","last_change":"2024-04-25T13:24:10.859008+0000","last_active":"2024-04-25T13:24:12.877190+0000","last_peered":"2024-04-25T13:24:12.877190+0000","last_clean":"2024-04-25T13:24:12.877190+0000","last_became_active":"2024-04-25T13:24:10.858350+0000","last_became_peered":"2024-04-25T13:24:10.858350+0000","last_unstale":"2024-04-25T13:24:12.877190+0000","last_undegraded":"2024-04-25T13:24:12.877190+0000","last_fullsized":"2024-04-25T13:24:12.877190+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.048592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.055016+0000","last_change":"2024-04-25T13:24:11.879185+0000","last_active":"2024-04-25T13:24:13.055016+0000","last_peered":"2024-04-25T13:24:13.055016+0000","last_clean":"2024-04-25T13:24:13.055016+0000","last_became_active":"2024-04-25T13:24:11.878808+0000","last_became_peered":"2024-04-25T13:24:11.878808+0000","last_unstale":"2024-04-25T13:24:13.055016+0000","last_undegraded":"2024-04-25T13:24:13.055016+0000","last_fullsized":"2024-04-25T13:24:13.055016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.408529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877331+0000","last_change":"2024-04-25T13:24:10.861742+0000","last_active":"2024-04-25T13:24:12.877331+0000","last_peered":"2024-04-25T13:24:12.877331+0000","last_clean":"2024-04-25T13:24:12.877331+0000","last_became_active":"2024-04-25T13:24:10.861302+0000","last_became_peered":"2024-04-25T13:24:10.861302+0000","last_unstale":"2024-04-25T13:24:12.877331+0000","last_undegraded":"2024-04-25T13:24:12.877331+0000","last_fullsized":"2024-04-25T13:24:12.877331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.470804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032284+0000","last_change":"2024-04-25T13:24:11.873872+0000","last_active":"2024-04-25T13:24:13.032284+0000","last_peered":"2024-04-25T13:24:13.032284+0000","last_clean":"2024-04-25T13:24:13.032284+0000","last_became_active":"2024-04-25T13:24:11.873341+0000","last_became_peered":"2024-04-25T13:24:11.873341+0000","last_unstale":"2024-04-25T13:24:13.032284+0000","last_undegraded":"2024-04-25T13:24:13.032284+0000","last_fullsized":"2024-04-25T13:24:13.032284+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:46.271742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877258+0000","last_change":"2024-04-25T13:24:10.856554+0000","last_active":"2024-04-25T13:24:12.877258+0000","last_peered":"2024-04-25T13:24:12.877258+0000","last_clean":"2024-04-25T13:24:12.877258+0000","last_became_active":"2024-04-25T13:24:10.856254+0000","last_became_peered":"2024-04-25T13:24:10.856254+0000","last_unstale":"2024-04-25T13:24:12.877258+0000","last_undegraded":"2024-04-25T13:24:12.877258+0000","last_fullsized":"2024-04-25T13:24:12.877258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:15.843985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031546+0000","last_change":"2024-04-25T13:24:11.891970+0000","last_active":"2024-04-25T13:24:13.031546+0000","last_peered":"2024-04-25T13:24:13.031546+0000","last_clean":"2024-04-25T13:24:13.031546+0000","last_became_active":"2024-04-25T13:24:11.891537+0000","last_became_peered":"2024-04-25T13:24:11.891537+0000","last_unstale":"2024-04-25T13:24:13.031546+0000","last_undegraded":"2024-04-25T13:24:13.031546+0000","last_fullsized":"2024-04-25T13:24:13.031546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.509206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031238+0000","last_change":"2024-04-25T13:24:10.857130+0000","last_active":"2024-04-25T13:24:13.031238+0000","last_peered":"2024-04-25T13:24:13.031238+0000","last_clean":"2024-04-25T13:24:13.031238+0000","last_became_active":"2024-04-25T13:24:10.856747+0000","last_became_peered":"2024-04-25T13:24:10.856747+0000","last_unstale":"2024-04-25T13:24:13.031238+0000","last_undegraded":"2024-04-25T13:24:13.031238+0000","last_fullsized":"2024-04-25T13:24:13.031238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:11.366223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877158+0000","last_change":"2024-04-25T13:24:11.873404+0000","last_active":"2024-04-25T13:24:12.877158+0000","last_peered":"2024-04-25T13:24:12.877158+0000","last_clean":"2024-04-25T13:24:12.877158+0000","last_became_active":"2024-04-25T13:24:11.873027+0000","last_became_peered":"2024-04-25T13:24:11.873027+0000","last_unstale":"2024-04-25T13:24:12.877158+0000","last_undegraded":"2024-04-25T13:24:12.877158+0000","last_fullsized":"2024-04-25T13:24:12.877158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:32.789300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031959+0000","last_change":"2024-04-25T13:24:10.853602+0000","last_active":"2024-04-25T13:24:13.031959+0000","last_peered":"2024-04-25T13:24:13.031959+0000","last_clean":"2024-04-25T13:24:13.031959+0000","last_became_active":"2024-04-25T13:24:10.853237+0000","last_became_peered":"2024-04-25T13:24:10.853237+0000","last_unstale":"2024-04-25T13:24:13.031959+0000","last_undegraded":"2024-04-25T13:24:13.031959+0000","last_fullsized":"2024-04-25T13:24:13.031959+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.575835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031318+0000","last_change":"2024-04-25T13:24:11.891757+0000","last_active":"2024-04-25T13:24:13.031318+0000","last_peered":"2024-04-25T13:24:13.031318+0000","last_clean":"2024-04-25T13:24:13.031318+0000","last_became_active":"2024-04-25T13:24:11.891362+0000","last_became_peered":"2024-04-25T13:24:11.891362+0000","last_unstale":"2024-04-25T13:24:13.031318+0000","last_undegraded":"2024-04-25T13:24:13.031318+0000","last_fullsized":"2024-04-25T13:24:13.031318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:44.202825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030789+0000","last_change":"2024-04-25T13:24:10.851042+0000","last_active":"2024-04-25T13:24:13.030789+0000","last_peered":"2024-04-25T13:24:13.030789+0000","last_clean":"2024-04-25T13:24:13.030789+0000","last_became_active":"2024-04-25T13:24:10.850619+0000","last_became_peered":"2024-04-25T13:24:10.850619+0000","last_unstale":"2024-04-25T13:24:13.030789+0000","last_undegraded":"2024-04-25T13:24:13.030789+0000","last_fullsized":"2024-04-25T13:24:13.030789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:23.577971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876876+0000","last_change":"2024-04-25T13:24:11.880170+0000","last_active":"2024-04-25T13:24:12.876876+0000","last_peered":"2024-04-25T13:24:12.876876+0000","last_clean":"2024-04-25T13:24:12.876876+0000","last_became_active":"2024-04-25T13:24:11.879802+0000","last_became_peered":"2024-04-25T13:24:11.879802+0000","last_unstale":"2024-04-25T13:24:12.876876+0000","last_undegraded":"2024-04-25T13:24:12.876876+0000","last_fullsized":"2024-04-25T13:24:12.876876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:25.231071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030717+0000","last_change":"2024-04-25T13:24:10.852667+0000","last_active":"2024-04-25T13:24:13.030717+0000","last_peered":"2024-04-25T13:24:13.030717+0000","last_clean":"2024-04-25T13:24:13.030717+0000","last_became_active":"2024-04-25T13:24:10.852222+0000","last_became_peered":"2024-04-25T13:24:10.852222+0000","last_unstale":"2024-04-25T13:24:13.030717+0000","last_undegraded":"2024-04-25T13:24:13.030717+0000","last_fullsized":"2024-04-25T13:24:13.030717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:06.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869303+0000","last_change":"2024-04-25T13:24:11.867984+0000","last_active":"2024-04-25T13:24:12.869303+0000","last_peered":"2024-04-25T13:24:12.869303+0000","last_clean":"2024-04-25T13:24:12.869303+0000","last_became_active":"2024-04-25T13:24:11.867517+0000","last_became_peered":"2024-04-25T13:24:11.867517+0000","last_unstale":"2024-04-25T13:24:12.869303+0000","last_undegraded":"2024-04-25T13:24:12.869303+0000","last_fullsized":"2024-04-25T13:24:12.869303+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.540573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032434+0000","last_change":"2024-04-25T13:24:10.855925+0000","last_active":"2024-04-25T13:24:13.032434+0000","last_peered":"2024-04-25T13:24:13.032434+0000","last_clean":"2024-04-25T13:24:13.032434+0000","last_became_active":"2024-04-25T13:24:10.855603+0000","last_became_peered":"2024-04-25T13:24:10.855603+0000","last_unstale":"2024-04-25T13:24:13.032434+0000","last_undegraded":"2024-04-25T13:24:13.032434+0000","last_fullsized":"2024-04-25T13:24:13.032434+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:26.796979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031354+0000","last_change":"2024-04-25T13:24:11.873683+0000","last_active":"2024-04-25T13:24:13.031354+0000","last_peered":"2024-04-25T13:24:13.031354+0000","last_clean":"2024-04-25T13:24:13.031354+0000","last_became_active":"2024-04-25T13:24:11.873307+0000","last_became_peered":"2024-04-25T13:24:11.873307+0000","last_unstale":"2024-04-25T13:24:13.031354+0000","last_undegraded":"2024-04-25T13:24:13.031354+0000","last_fullsized":"2024-04-25T13:24:13.031354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:44.305349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872528+0000","last_change":"2024-04-25T13:24:10.857124+0000","last_active":"2024-04-25T13:24:12.872528+0000","last_peered":"2024-04-25T13:24:12.872528+0000","last_clean":"2024-04-25T13:24:12.872528+0000","last_became_active":"2024-04-25T13:24:10.856396+0000","last_became_peered":"2024-04-25T13:24:10.856396+0000","last_unstale":"2024-04-25T13:24:12.872528+0000","last_undegraded":"2024-04-25T13:24:12.872528+0000","last_fullsized":"2024-04-25T13:24:12.872528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:58.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.055015+0000","last_change":"2024-04-25T13:24:11.882305+0000","last_active":"2024-04-25T13:24:13.055015+0000","last_peered":"2024-04-25T13:24:13.055015+0000","last_clean":"2024-04-25T13:24:13.055015+0000","last_became_active":"2024-04-25T13:24:11.881976+0000","last_became_peered":"2024-04-25T13:24:11.881976+0000","last_unstale":"2024-04-25T13:24:13.055015+0000","last_undegraded":"2024-04-25T13:24:13.055015+0000","last_fullsized":"2024-04-25T13:24:13.055015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:46.254871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'3","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031520+0000","last_change":"2024-04-25T13:24:10.864302+0000","last_active":"2024-04-25T13:24:13.031520+0000","last_peered":"2024-04-25T13:24:13.031520+0000","last_clean":"2024-04-25T13:24:13.031520+0000","last_became_active":"2024-04-25T13:24:10.863955+0000","last_became_peered":"2024-04-25T13:24:10.863955+0000","last_unstale":"2024-04-25T13:24:13.031520+0000","last_undegraded":"2024-04-25T13:24:13.031520+0000","last_fullsized":"2024-04-25T13:24:13.031520+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:35.394778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030883+0000","last_change":"2024-04-25T13:24:11.876293+0000","last_active":"2024-04-25T13:24:13.030883+0000","last_peered":"2024-04-25T13:24:13.030883+0000","last_clean":"2024-04-25T13:24:13.030883+0000","last_became_active":"2024-04-25T13:24:11.875927+0000","last_became_peered":"2024-04-25T13:24:11.875927+0000","last_unstale":"2024-04-25T13:24:13.030883+0000","last_undegraded":"2024-04-25T13:24:13.030883+0000","last_fullsized":"2024-04-25T13:24:13.030883+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:47.884328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:14.892043+0000","last_change":"2024-04-25T13:24:10.857046+0000","last_active":"2024-04-25T13:24:14.892043+0000","last_peered":"2024-04-25T13:24:14.892043+0000","last_clean":"2024-04-25T13:24:14.892043+0000","last_became_active":"2024-04-25T13:24:10.856724+0000","last_became_peered":"2024-04-25T13:24:10.856724+0000","last_unstale":"2024-04-25T13:24:14.892043+0000","last_undegraded":"2024-04-25T13:24:14.892043+0000","last_fullsized":"2024-04-25T13:24:14.892043+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:55.913961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877257+0000","last_change":"2024-04-25T13:24:11.886845+0000","last_active":"2024-04-25T13:24:12.877257+0000","last_peered":"2024-04-25T13:24:12.877257+0000","last_clean":"2024-04-25T13:24:12.877257+0000","last_became_active":"2024-04-25T13:24:11.885457+0000","last_became_peered":"2024-04-25T13:24:11.885457+0000","last_unstale":"2024-04-25T13:24:12.877257+0000","last_undegraded":"2024-04-25T13:24:12.877257+0000","last_fullsized":"2024-04-25T13:24:12.877257+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.134026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870305+0000","last_change":"2024-04-25T13:24:10.865292+0000","last_active":"2024-04-25T13:24:12.870305+0000","last_peered":"2024-04-25T13:24:12.870305+0000","last_clean":"2024-04-25T13:24:12.870305+0000","last_became_active":"2024-04-25T13:24:10.864940+0000","last_became_peered":"2024-04-25T13:24:10.864940+0000","last_unstale":"2024-04-25T13:24:12.870305+0000","last_undegraded":"2024-04-25T13:24:12.870305+0000","last_fullsized":"2024-04-25T13:24:12.870305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:54.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030956+0000","last_change":"2024-04-25T13:24:11.885158+0000","last_active":"2024-04-25T13:24:13.030956+0000","last_peered":"2024-04-25T13:24:13.030956+0000","last_clean":"2024-04-25T13:24:13.030956+0000","last_became_active":"2024-04-25T13:24:11.884367+0000","last_became_peered":"2024-04-25T13:24:11.884367+0000","last_unstale":"2024-04-25T13:24:13.030956+0000","last_undegraded":"2024-04-25T13:24:13.030956+0000","last_fullsized":"2024-04-25T13:24:13.030956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:48.244922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031218+0000","last_change":"2024-04-25T13:24:10.854269+0000","last_active":"2024-04-25T13:24:13.031218+0000","last_peered":"2024-04-25T13:24:13.031218+0000","last_clean":"2024-04-25T13:24:13.031218+0000","last_became_active":"2024-04-25T13:24:10.853968+0000","last_became_peered":"2024-04-25T13:24:10.853968+0000","last_unstale":"2024-04-25T13:24:13.031218+0000","last_undegraded":"2024-04-25T13:24:13.031218+0000","last_fullsized":"2024-04-25T13:24:13.031218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:42.771743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872675+0000","last_change":"2024-04-25T13:24:11.887468+0000","last_active":"2024-04-25T13:24:12.872675+0000","last_peered":"2024-04-25T13:24:12.872675+0000","last_clean":"2024-04-25T13:24:12.872675+0000","last_became_active":"2024-04-25T13:24:11.887097+0000","last_became_peered":"2024-04-25T13:24:11.887097+0000","last_unstale":"2024-04-25T13:24:12.872675+0000","last_undegraded":"2024-04-25T13:24:12.872675+0000","last_fullsized":"2024-04-25T13:24:12.872675+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:23.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030997+0000","last_change":"2024-04-25T13:24:10.854484+0000","last_active":"2024-04-25T13:24:13.030997+0000","last_peered":"2024-04-25T13:24:13.030997+0000","last_clean":"2024-04-25T13:24:13.030997+0000","last_became_active":"2024-04-25T13:24:10.854152+0000","last_became_peered":"2024-04-25T13:24:10.854152+0000","last_unstale":"2024-04-25T13:24:13.030997+0000","last_undegraded":"2024-04-25T13:24:13.030997+0000","last_fullsized":"2024-04-25T13:24:13.030997+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:25.940414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872170+0000","last_change":"2024-04-25T13:24:11.880661+0000","last_active":"2024-04-25T13:24:12.872170+0000","last_peered":"2024-04-25T13:24:12.872170+0000","last_clean":"2024-04-25T13:24:12.872170+0000","last_became_active":"2024-04-25T13:24:11.880099+0000","last_became_peered":"2024-04-25T13:24:11.880099+0000","last_unstale":"2024-04-25T13:24:12.872170+0000","last_undegraded":"2024-04-25T13:24:12.872170+0000","last_fullsized":"2024-04-25T13:24:12.872170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876763+0000","last_change":"2024-04-25T13:24:10.863356+0000","last_active":"2024-04-25T13:24:12.876763+0000","last_peered":"2024-04-25T13:24:12.876763+0000","last_clean":"2024-04-25T13:24:12.876763+0000","last_became_active":"2024-04-25T13:24:10.863024+0000","last_became_peered":"2024-04-25T13:24:10.863024+0000","last_unstale":"2024-04-25T13:24:12.876763+0000","last_undegraded":"2024-04-25T13:24:12.876763+0000","last_fullsized":"2024-04-25T13:24:12.876763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:11.520794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030845+0000","last_change":"2024-04-25T13:24:11.891787+0000","last_active":"2024-04-25T13:24:13.030845+0000","last_peered":"2024-04-25T13:24:13.030845+0000","last_clean":"2024-04-25T13:24:13.030845+0000","last_became_active":"2024-04-25T13:24:11.891466+0000","last_became_peered":"2024-04-25T13:24:11.891466+0000","last_unstale":"2024-04-25T13:24:13.030845+0000","last_undegraded":"2024-04-25T13:24:13.030845+0000","last_fullsized":"2024-04-25T13:24:13.030845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:11.200558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877294+0000","last_change":"2024-04-25T13:24:10.852320+0000","last_active":"2024-04-25T13:24:12.877294+0000","last_peered":"2024-04-25T13:24:12.877294+0000","last_clean":"2024-04-25T13:24:12.877294+0000","last_became_active":"2024-04-25T13:24:10.852007+0000","last_became_peered":"2024-04-25T13:24:10.852007+0000","last_unstale":"2024-04-25T13:24:12.877294+0000","last_undegraded":"2024-04-25T13:24:12.877294+0000","last_fullsized":"2024-04-25T13:24:12.877294+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.582461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031489+0000","last_change":"2024-04-25T13:24:11.875174+0000","last_active":"2024-04-25T13:24:13.031489+0000","last_peered":"2024-04-25T13:24:13.031489+0000","last_clean":"2024-04-25T13:24:13.031489+0000","last_became_active":"2024-04-25T13:24:11.874813+0000","last_became_peered":"2024-04-25T13:24:11.874813+0000","last_unstale":"2024-04-25T13:24:13.031489+0000","last_undegraded":"2024-04-25T13:24:13.031489+0000","last_fullsized":"2024-04-25T13:24:13.031489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:22.737412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869343+0000","last_change":"2024-04-25T13:24:10.857715+0000","last_active":"2024-04-25T13:24:12.869343+0000","last_peered":"2024-04-25T13:24:12.869343+0000","last_clean":"2024-04-25T13:24:12.869343+0000","last_became_active":"2024-04-25T13:24:10.857398+0000","last_became_peered":"2024-04-25T13:24:10.857398+0000","last_unstale":"2024-04-25T13:24:12.869343+0000","last_undegraded":"2024-04-25T13:24:12.869343+0000","last_fullsized":"2024-04-25T13:24:12.869343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:58.189652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032597+0000","last_change":"2024-04-25T13:24:11.875065+0000","last_active":"2024-04-25T13:24:13.032597+0000","last_peered":"2024-04-25T13:24:13.032597+0000","last_clean":"2024-04-25T13:24:13.032597+0000","last_became_active":"2024-04-25T13:24:11.874433+0000","last_became_peered":"2024-04-25T13:24:11.874433+0000","last_unstale":"2024-04-25T13:24:13.032597+0000","last_undegraded":"2024-04-25T13:24:13.032597+0000","last_fullsized":"2024-04-25T13:24:13.032597+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:07.430007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872227+0000","last_change":"2024-04-25T13:24:10.850250+0000","last_active":"2024-04-25T13:24:12.872227+0000","last_peered":"2024-04-25T13:24:12.872227+0000","last_clean":"2024-04-25T13:24:12.872227+0000","last_became_active":"2024-04-25T13:24:10.849773+0000","last_became_peered":"2024-04-25T13:24:10.849773+0000","last_unstale":"2024-04-25T13:24:12.872227+0000","last_undegraded":"2024-04-25T13:24:12.872227+0000","last_fullsized":"2024-04-25T13:24:12.872227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:59.609421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032615+0000","last_change":"2024-04-25T13:24:11.878625+0000","last_active":"2024-04-25T13:24:13.032615+0000","last_peered":"2024-04-25T13:24:13.032615+0000","last_clean":"2024-04-25T13:24:13.032615+0000","last_became_active":"2024-04-25T13:24:11.878311+0000","last_became_peered":"2024-04-25T13:24:11.878311+0000","last_unstale":"2024-04-25T13:24:13.032615+0000","last_undegraded":"2024-04-25T13:24:13.032615+0000","last_fullsized":"2024-04-25T13:24:13.032615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.363441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031060+0000","last_change":"2024-04-25T13:24:11.887874+0000","last_active":"2024-04-25T13:24:13.031060+0000","last_peered":"2024-04-25T13:24:13.031060+0000","last_clean":"2024-04-25T13:24:13.031060+0000","last_became_active":"2024-04-25T13:24:11.887558+0000","last_became_peered":"2024-04-25T13:24:11.887558+0000","last_unstale":"2024-04-25T13:24:13.031060+0000","last_undegraded":"2024-04-25T13:24:13.031060+0000","last_fullsized":"2024-04-25T13:24:13.031060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:49.861447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030655+0000","last_change":"2024-04-25T13:24:10.859643+0000","last_active":"2024-04-25T13:24:13.030655+0000","last_peered":"2024-04-25T13:24:13.030655+0000","last_clean":"2024-04-25T13:24:13.030655+0000","last_became_active":"2024-04-25T13:24:10.859312+0000","last_became_peered":"2024-04-25T13:24:10.859312+0000","last_unstale":"2024-04-25T13:24:13.030655+0000","last_undegraded":"2024-04-25T13:24:13.030655+0000","last_fullsized":"2024-04-25T13:24:13.030655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.513203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877036+0000","last_change":"2024-04-25T13:24:11.883821+0000","last_active":"2024-04-25T13:24:12.877036+0000","last_peered":"2024-04-25T13:24:12.877036+0000","last_clean":"2024-04-25T13:24:12.877036+0000","last_became_active":"2024-04-25T13:24:11.883063+0000","last_became_peered":"2024-04-25T13:24:11.883063+0000","last_unstale":"2024-04-25T13:24:12.877036+0000","last_undegraded":"2024-04-25T13:24:12.877036+0000","last_fullsized":"2024-04-25T13:24:12.877036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.845694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031477+0000","last_change":"2024-04-25T13:24:10.851525+0000","last_active":"2024-04-25T13:24:13.031477+0000","last_peered":"2024-04-25T13:24:13.031477+0000","last_clean":"2024-04-25T13:24:13.031477+0000","last_became_active":"2024-04-25T13:24:10.851171+0000","last_became_peered":"2024-04-25T13:24:10.851171+0000","last_unstale":"2024-04-25T13:24:13.031477+0000","last_undegraded":"2024-04-25T13:24:13.031477+0000","last_fullsized":"2024-04-25T13:24:13.031477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872078+0000","last_change":"2024-04-25T13:22:58.125848+0000","last_active":"2024-04-25T13:24:12.872078+0000","last_peered":"2024-04-25T13:24:12.872078+0000","last_clean":"2024-04-25T13:24:12.872078+0000","last_became_active":"2024-04-25T13:22:58.125374+0000","last_became_peered":"2024-04-25T13:22:58.125374+0000","last_unstale":"2024-04-25T13:24:12.872078+0000","last_undegraded":"2024-04-25T13:24:12.872078+0000","last_fullsized":"2024-04-25T13:24:12.872078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_clean_scrub_stamp":"2024-04-25T13:19:36.251667+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.304605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030780+0000","last_change":"2024-04-25T13:24:11.883648+0000","last_active":"2024-04-25T13:24:13.030780+0000","last_peered":"2024-04-25T13:24:13.030780+0000","last_clean":"2024-04-25T13:24:13.030780+0000","last_became_active":"2024-04-25T13:24:11.879538+0000","last_became_peered":"2024-04-25T13:24:11.879538+0000","last_unstale":"2024-04-25T13:24:13.030780+0000","last_undegraded":"2024-04-25T13:24:13.030780+0000","last_fullsized":"2024-04-25T13:24:13.030780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872254+0000","last_change":"2024-04-25T13:24:10.851697+0000","last_active":"2024-04-25T13:24:12.872254+0000","last_peered":"2024-04-25T13:24:12.872254+0000","last_clean":"2024-04-25T13:24:12.872254+0000","last_became_active":"2024-04-25T13:24:10.851379+0000","last_became_peered":"2024-04-25T13:24:10.851379+0000","last_unstale":"2024-04-25T13:24:12.872254+0000","last_undegraded":"2024-04-25T13:24:12.872254+0000","last_fullsized":"2024-04-25T13:24:12.872254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:13.104612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877120+0000","last_change":"2024-04-25T13:24:10.864221+0000","last_active":"2024-04-25T13:24:12.877120+0000","last_peered":"2024-04-25T13:24:12.877120+0000","last_clean":"2024-04-25T13:24:12.877120+0000","last_became_active":"2024-04-25T13:24:10.863892+0000","last_became_peered":"2024-04-25T13:24:10.863892+0000","last_unstale":"2024-04-25T13:24:12.877120+0000","last_undegraded":"2024-04-25T13:24:12.877120+0000","last_fullsized":"2024-04-25T13:24:12.877120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:51.229865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031120+0000","last_change":"2024-04-25T13:24:11.878257+0000","last_active":"2024-04-25T13:24:13.031120+0000","last_peered":"2024-04-25T13:24:13.031120+0000","last_clean":"2024-04-25T13:24:13.031120+0000","last_became_active":"2024-04-25T13:24:11.877926+0000","last_became_peered":"2024-04-25T13:24:11.877926+0000","last_unstale":"2024-04-25T13:24:13.031120+0000","last_undegraded":"2024-04-25T13:24:13.031120+0000","last_fullsized":"2024-04-25T13:24:13.031120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:32.873011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.869522+0000","last_change":"2024-04-25T13:24:10.860867+0000","last_active":"2024-04-25T13:24:12.869522+0000","last_peered":"2024-04-25T13:24:12.869522+0000","last_clean":"2024-04-25T13:24:12.869522+0000","last_became_active":"2024-04-25T13:24:10.860566+0000","last_became_peered":"2024-04-25T13:24:10.860566+0000","last_unstale":"2024-04-25T13:24:12.869522+0000","last_undegraded":"2024-04-25T13:24:12.869522+0000","last_fullsized":"2024-04-25T13:24:12.869522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.340184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877157+0000","last_change":"2024-04-25T13:24:11.884573+0000","last_active":"2024-04-25T13:24:12.877157+0000","last_peered":"2024-04-25T13:24:12.877157+0000","last_clean":"2024-04-25T13:24:12.877157+0000","last_became_active":"2024-04-25T13:24:11.884245+0000","last_became_peered":"2024-04-25T13:24:11.884245+0000","last_unstale":"2024-04-25T13:24:12.877157+0000","last_undegraded":"2024-04-25T13:24:12.877157+0000","last_fullsized":"2024-04-25T13:24:12.877157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:12.555396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031355+0000","last_change":"2024-04-25T13:24:10.854441+0000","last_active":"2024-04-25T13:24:13.031355+0000","last_peered":"2024-04-25T13:24:13.031355+0000","last_clean":"2024-04-25T13:24:13.031355+0000","last_became_active":"2024-04-25T13:24:10.854086+0000","last_became_peered":"2024-04-25T13:24:10.854086+0000","last_unstale":"2024-04-25T13:24:13.031355+0000","last_undegraded":"2024-04-25T13:24:13.031355+0000","last_fullsized":"2024-04-25T13:24:13.031355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:17.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054861+0000","last_change":"2024-04-25T13:24:11.880494+0000","last_active":"2024-04-25T13:24:13.054861+0000","last_peered":"2024-04-25T13:24:13.054861+0000","last_clean":"2024-04-25T13:24:13.054861+0000","last_became_active":"2024-04-25T13:24:11.880074+0000","last_became_peered":"2024-04-25T13:24:11.880074+0000","last_unstale":"2024-04-25T13:24:13.054861+0000","last_undegraded":"2024-04-25T13:24:13.054861+0000","last_fullsized":"2024-04-25T13:24:13.054861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:50.993165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032176+0000","last_change":"2024-04-25T13:24:10.863065+0000","last_active":"2024-04-25T13:24:13.032176+0000","last_peered":"2024-04-25T13:24:13.032176+0000","last_clean":"2024-04-25T13:24:13.032176+0000","last_became_active":"2024-04-25T13:24:10.862793+0000","last_became_peered":"2024-04-25T13:24:10.862793+0000","last_unstale":"2024-04-25T13:24:13.032176+0000","last_undegraded":"2024-04-25T13:24:13.032176+0000","last_fullsized":"2024-04-25T13:24:13.032176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:03.719745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876929+0000","last_change":"2024-04-25T13:24:11.878770+0000","last_active":"2024-04-25T13:24:12.876929+0000","last_peered":"2024-04-25T13:24:12.876929+0000","last_clean":"2024-04-25T13:24:12.876929+0000","last_became_active":"2024-04-25T13:24:11.878424+0000","last_became_peered":"2024-04-25T13:24:11.878424+0000","last_unstale":"2024-04-25T13:24:12.876929+0000","last_undegraded":"2024-04-25T13:24:12.876929+0000","last_fullsized":"2024-04-25T13:24:12.876929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:25.016200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876720+0000","last_change":"2024-04-25T13:24:10.857246+0000","last_active":"2024-04-25T13:24:12.876720+0000","last_peered":"2024-04-25T13:24:12.876720+0000","last_clean":"2024-04-25T13:24:12.876720+0000","last_became_active":"2024-04-25T13:24:10.856922+0000","last_became_peered":"2024-04-25T13:24:10.856922+0000","last_unstale":"2024-04-25T13:24:12.876720+0000","last_undegraded":"2024-04-25T13:24:12.876720+0000","last_fullsized":"2024-04-25T13:24:12.876720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:08.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030750+0000","last_change":"2024-04-25T13:24:11.873225+0000","last_active":"2024-04-25T13:24:13.030750+0000","last_peered":"2024-04-25T13:24:13.030750+0000","last_clean":"2024-04-25T13:24:13.030750+0000","last_became_active":"2024-04-25T13:24:11.872855+0000","last_became_peered":"2024-04-25T13:24:11.872855+0000","last_unstale":"2024-04-25T13:24:13.030750+0000","last_undegraded":"2024-04-25T13:24:13.030750+0000","last_fullsized":"2024-04-25T13:24:13.030750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.523621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870070+0000","last_change":"2024-04-25T13:24:10.863165+0000","last_active":"2024-04-25T13:24:12.870070+0000","last_peered":"2024-04-25T13:24:12.870070+0000","last_clean":"2024-04-25T13:24:12.870070+0000","last_became_active":"2024-04-25T13:24:10.862596+0000","last_became_peered":"2024-04-25T13:24:10.862596+0000","last_unstale":"2024-04-25T13:24:12.870070+0000","last_undegraded":"2024-04-25T13:24:12.870070+0000","last_fullsized":"2024-04-25T13:24:12.870070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:51.414158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877091+0000","last_change":"2024-04-25T13:24:11.885852+0000","last_active":"2024-04-25T13:24:12.877091+0000","last_peered":"2024-04-25T13:24:12.877091+0000","last_clean":"2024-04-25T13:24:12.877091+0000","last_became_active":"2024-04-25T13:24:11.885508+0000","last_became_peered":"2024-04-25T13:24:11.885508+0000","last_unstale":"2024-04-25T13:24:12.877091+0000","last_undegraded":"2024-04-25T13:24:12.877091+0000","last_fullsized":"2024-04-25T13:24:12.877091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.575351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877007+0000","last_change":"2024-04-25T13:24:10.861701+0000","last_active":"2024-04-25T13:24:12.877007+0000","last_peered":"2024-04-25T13:24:12.877007+0000","last_clean":"2024-04-25T13:24:12.877007+0000","last_became_active":"2024-04-25T13:24:10.861356+0000","last_became_peered":"2024-04-25T13:24:10.861356+0000","last_unstale":"2024-04-25T13:24:12.877007+0000","last_undegraded":"2024-04-25T13:24:12.877007+0000","last_fullsized":"2024-04-25T13:24:12.877007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:24.300835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872496+0000","last_change":"2024-04-25T13:24:11.883977+0000","last_active":"2024-04-25T13:24:12.872496+0000","last_peered":"2024-04-25T13:24:12.872496+0000","last_clean":"2024-04-25T13:24:12.872496+0000","last_became_active":"2024-04-25T13:24:11.883401+0000","last_became_peered":"2024-04-25T13:24:11.883401+0000","last_unstale":"2024-04-25T13:24:12.872496+0000","last_undegraded":"2024-04-25T13:24:12.872496+0000","last_fullsized":"2024-04-25T13:24:12.872496+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:04.291865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870384+0000","last_change":"2024-04-25T13:24:10.858303+0000","last_active":"2024-04-25T13:24:12.870384+0000","last_peered":"2024-04-25T13:24:12.870384+0000","last_clean":"2024-04-25T13:24:12.870384+0000","last_became_active":"2024-04-25T13:24:10.857956+0000","last_became_peered":"2024-04-25T13:24:10.857956+0000","last_unstale":"2024-04-25T13:24:12.870384+0000","last_undegraded":"2024-04-25T13:24:12.870384+0000","last_fullsized":"2024-04-25T13:24:12.870384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.753759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872720+0000","last_change":"2024-04-25T13:24:11.882367+0000","last_active":"2024-04-25T13:24:12.872720+0000","last_peered":"2024-04-25T13:24:12.872720+0000","last_clean":"2024-04-25T13:24:12.872720+0000","last_became_active":"2024-04-25T13:24:11.880495+0000","last_became_peered":"2024-04-25T13:24:11.880495+0000","last_unstale":"2024-04-25T13:24:12.872720+0000","last_undegraded":"2024-04-25T13:24:12.872720+0000","last_fullsized":"2024-04-25T13:24:12.872720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.031326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031381+0000","last_change":"2024-04-25T13:24:10.854984+0000","last_active":"2024-04-25T13:24:13.031381+0000","last_peered":"2024-04-25T13:24:13.031381+0000","last_clean":"2024-04-25T13:24:13.031381+0000","last_became_active":"2024-04-25T13:24:10.854691+0000","last_became_peered":"2024-04-25T13:24:10.854691+0000","last_unstale":"2024-04-25T13:24:13.031381+0000","last_undegraded":"2024-04-25T13:24:13.031381+0000","last_fullsized":"2024-04-25T13:24:13.031381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:55.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872420+0000","last_change":"2024-04-25T13:24:11.881329+0000","last_active":"2024-04-25T13:24:12.872420+0000","last_peered":"2024-04-25T13:24:12.872420+0000","last_clean":"2024-04-25T13:24:12.872420+0000","last_became_active":"2024-04-25T13:24:11.880958+0000","last_became_peered":"2024-04-25T13:24:11.880958+0000","last_unstale":"2024-04-25T13:24:12.872420+0000","last_undegraded":"2024-04-25T13:24:12.872420+0000","last_fullsized":"2024-04-25T13:24:12.872420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:54.720129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030876+0000","last_change":"2024-04-25T13:24:10.861501+0000","last_active":"2024-04-25T13:24:13.030876+0000","last_peered":"2024-04-25T13:24:13.030876+0000","last_clean":"2024-04-25T13:24:13.030876+0000","last_became_active":"2024-04-25T13:24:10.861155+0000","last_became_peered":"2024-04-25T13:24:10.861155+0000","last_unstale":"2024-04-25T13:24:13.030876+0000","last_undegraded":"2024-04-25T13:24:13.030876+0000","last_fullsized":"2024-04-25T13:24:13.030876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:17.306214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.872835+0000","last_change":"2024-04-25T13:24:11.879779+0000","last_active":"2024-04-25T13:24:12.872835+0000","last_peered":"2024-04-25T13:24:12.872835+0000","last_clean":"2024-04-25T13:24:12.872835+0000","last_became_active":"2024-04-25T13:24:11.879285+0000","last_became_peered":"2024-04-25T13:24:11.879285+0000","last_unstale":"2024-04-25T13:24:12.872835+0000","last_undegraded":"2024-04-25T13:24:12.872835+0000","last_fullsized":"2024-04-25T13:24:12.872835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.799094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031003+0000","last_change":"2024-04-25T13:24:11.871059+0000","last_active":"2024-04-25T13:24:13.031003+0000","last_peered":"2024-04-25T13:24:13.031003+0000","last_clean":"2024-04-25T13:24:13.031003+0000","last_became_active":"2024-04-25T13:24:11.869530+0000","last_became_peered":"2024-04-25T13:24:11.869530+0000","last_unstale":"2024-04-25T13:24:13.031003+0000","last_undegraded":"2024-04-25T13:24:13.031003+0000","last_fullsized":"2024-04-25T13:24:13.031003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.095146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870339+0000","last_change":"2024-04-25T13:24:11.876162+0000","last_active":"2024-04-25T13:24:12.870339+0000","last_peered":"2024-04-25T13:24:12.870339+0000","last_clean":"2024-04-25T13:24:12.870339+0000","last_became_active":"2024-04-25T13:24:11.875747+0000","last_became_peered":"2024-04-25T13:24:11.875747+0000","last_unstale":"2024-04-25T13:24:12.870339+0000","last_undegraded":"2024-04-25T13:24:12.870339+0000","last_fullsized":"2024-04-25T13:24:12.870339+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:07.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870214+0000","last_change":"2024-04-25T13:24:11.887477+0000","last_active":"2024-04-25T13:24:12.870214+0000","last_peered":"2024-04-25T13:24:12.870214+0000","last_clean":"2024-04-25T13:24:12.870214+0000","last_became_active":"2024-04-25T13:24:11.886951+0000","last_became_peered":"2024-04-25T13:24:11.886951+0000","last_unstale":"2024-04-25T13:24:12.870214+0000","last_undegraded":"2024-04-25T13:24:12.870214+0000","last_fullsized":"2024-04-25T13:24:12.870214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:28.865817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.054680+0000","last_change":"2024-04-25T13:24:11.882545+0000","last_active":"2024-04-25T13:24:13.054680+0000","last_peered":"2024-04-25T13:24:13.054680+0000","last_clean":"2024-04-25T13:24:13.054680+0000","last_became_active":"2024-04-25T13:24:11.881461+0000","last_became_peered":"2024-04-25T13:24:11.881461+0000","last_unstale":"2024-04-25T13:24:13.054680+0000","last_undegraded":"2024-04-25T13:24:13.054680+0000","last_fullsized":"2024-04-25T13:24:13.054680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:26.823813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031085+0000","last_change":"2024-04-25T13:24:11.876929+0000","last_active":"2024-04-25T13:24:13.031085+0000","last_peered":"2024-04-25T13:24:13.031085+0000","last_clean":"2024-04-25T13:24:13.031085+0000","last_became_active":"2024-04-25T13:24:11.876578+0000","last_became_peered":"2024-04-25T13:24:11.876578+0000","last_unstale":"2024-04-25T13:24:13.031085+0000","last_undegraded":"2024-04-25T13:24:13.031085+0000","last_fullsized":"2024-04-25T13:24:13.031085+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.621282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031200+0000","last_change":"2024-04-25T13:24:11.869478+0000","last_active":"2024-04-25T13:24:13.031200+0000","last_peered":"2024-04-25T13:24:13.031200+0000","last_clean":"2024-04-25T13:24:13.031200+0000","last_became_active":"2024-04-25T13:24:11.869104+0000","last_became_peered":"2024-04-25T13:24:11.869104+0000","last_unstale":"2024-04-25T13:24:13.031200+0000","last_undegraded":"2024-04-25T13:24:13.031200+0000","last_fullsized":"2024-04-25T13:24:13.031200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:46.865868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.870664+0000","last_change":"2024-04-25T13:24:11.874008+0000","last_active":"2024-04-25T13:24:12.870664+0000","last_peered":"2024-04-25T13:24:12.870664+0000","last_clean":"2024-04-25T13:24:12.870664+0000","last_became_active":"2024-04-25T13:24:11.873576+0000","last_became_peered":"2024-04-25T13:24:11.873576+0000","last_unstale":"2024-04-25T13:24:12.870664+0000","last_undegraded":"2024-04-25T13:24:12.870664+0000","last_fullsized":"2024-04-25T13:24:12.870664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:14.395329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.877099+0000","last_change":"2024-04-25T13:24:11.880749+0000","last_active":"2024-04-25T13:24:12.877099+0000","last_peered":"2024-04-25T13:24:12.877099+0000","last_clean":"2024-04-25T13:24:12.877099+0000","last_became_active":"2024-04-25T13:24:11.880421+0000","last_became_peered":"2024-04-25T13:24:11.880421+0000","last_unstale":"2024-04-25T13:24:12.877099+0000","last_undegraded":"2024-04-25T13:24:12.877099+0000","last_fullsized":"2024-04-25T13:24:12.877099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:44.001354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.030844+0000","last_change":"2024-04-25T13:24:11.885922+0000","last_active":"2024-04-25T13:24:13.030844+0000","last_peered":"2024-04-25T13:24:13.030844+0000","last_clean":"2024-04-25T13:24:13.030844+0000","last_became_active":"2024-04-25T13:24:11.885611+0000","last_became_peered":"2024-04-25T13:24:11.885611+0000","last_unstale":"2024-04-25T13:24:13.030844+0000","last_undegraded":"2024-04-25T13:24:13.030844+0000","last_fullsized":"2024-04-25T13:24:13.030844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:12.876857+0000","last_change":"2024-04-25T13:24:11.877912+0000","last_active":"2024-04-25T13:24:12.876857+0000","last_peered":"2024-04-25T13:24:12.876857+0000","last_clean":"2024-04-25T13:24:12.876857+0000","last_became_active":"2024-04-25T13:24:11.877139+0000","last_became_peered":"2024-04-25T13:24:11.877139+0000","last_unstale":"2024-04-25T13:24:12.876857+0000","last_undegraded":"2024-04-25T13:24:12.876857+0000","last_fullsized":"2024-04-25T13:24:12.876857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:45.618556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.032263+0000","last_change":"2024-04-25T13:24:11.889267+0000","last_active":"2024-04-25T13:24:13.032263+0000","last_peered":"2024-04-25T13:24:13.032263+0000","last_clean":"2024-04-25T13:24:13.032263+0000","last_became_active":"2024-04-25T13:24:11.888875+0000","last_became_peered":"2024-04-25T13:24:11.888875+0000","last_unstale":"2024-04-25T13:24:13.032263+0000","last_undegraded":"2024-04-25T13:24:13.032263+0000","last_fullsized":"2024-04-25T13:24:13.032263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:41.871075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031125+0000","last_change":"2024-04-25T13:24:11.890664+0000","last_active":"2024-04-25T13:24:13.031125+0000","last_peered":"2024-04-25T13:24:13.031125+0000","last_clean":"2024-04-25T13:24:13.031125+0000","last_became_active":"2024-04-25T13:24:11.890124+0000","last_became_peered":"2024-04-25T13:24:11.890124+0000","last_unstale":"2024-04-25T13:24:13.031125+0000","last_undegraded":"2024-04-25T13:24:13.031125+0000","last_fullsized":"2024-04-25T13:24:13.031125+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:55.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T13:24:13.031031+0000","last_change":"2024-04-25T13:24:11.876201+0000","last_active":"2024-04-25T13:24:13.031031+0000","last_peered":"2024-04-25T13:24:13.031031+0000","last_clean":"2024-04-25T13:24:13.031031+0000","last_became_active":"2024-04-25T13:24:11.875783+0000","last_became_peered":"2024-04-25T13:24:11.875783+0000","last_unstale":"2024-04-25T13:24:13.031031+0000","last_undegraded":"2024-04-25T13:24:13.031031+0000","last_fullsized":"2024-04-25T13:24:13.031031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.264576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":9,"num_read_kb":8,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":13830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":954368,"data_stored":456350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514515,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1572864,"data_stored":1048524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678040,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":958464,"data_stored":456932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939678,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":456954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234019,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712452,"statfs":{"total":95995035648,"available":95961550848,"internally_reserved":0,"allocated":1552384,"data_stored":1047300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":954368,"data_stored":456350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":460368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":456440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":457996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117046,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":962560,"data_stored":456454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":456372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691884,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1560576,"data_stored":1046900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:24:19.070 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:24:19.071 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T13:24:19.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T13:24:19.351 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:18 smithi069 ceph-mon[32534]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:18 smithi069 ceph-mon[32534]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T13:24:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:18 smithi069 ceph-mon[32534]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:24:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:18 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/831991281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:18 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2357652895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:18 smithi044 ceph-mon[32590]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:24:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:18 smithi044 ceph-mon[32590]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T13:24:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:18 smithi044 ceph-mon[32590]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T13:24:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:18 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/831991281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:24:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:18 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2357652895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:20.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:19 smithi037 ceph-mon[29204]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:20.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:19 smithi037 ceph-mon[29204]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:20.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:19 smithi069 ceph-mon[32534]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:20.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:19 smithi069 ceph-mon[32534]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:20.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:19 smithi044 ceph-mon[32590]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:20.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:19 smithi044 ceph-mon[32590]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:24:21.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:20 smithi037 ceph-mon[29204]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:24:21.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 -- 172.21.15.37:0/3641572460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14108810 msgr2=0x7fba1410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/3641572460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14108810 0x7fba1410ac20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00b0a0 tx=0x7fba0c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 -- 172.21.15.37:0/3641572460 shutdown_connections 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/3641572460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba141035b0 0x7fba141059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/3641572460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba14108810 0x7fba1410ac20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/3641572460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 0x7fba141082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 -- 172.21.15.37:0/3641572460 >> 172.21.15.37:0/3641572460 conn(0x7fba140fcf60 msgr2=0x7fba140ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 -- 172.21.15.37:0/3641572460 shutdown_connections 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.125+0000 7fba1ab6e700 1 -- 172.21.15.37:0/3641572460 wait complete. 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 Processor -- start 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 -- start start 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 0x7fba14101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:21.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba14108810 0x7fba14101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba1410e520 con 0x7fba141035b0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fba1410e3a0 con 0x7fba14108810 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1ab6e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba1410e6a0 con 0x7fba14105ee0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42542/0 (socket says 172.21.15.37:42542) 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 -- 172.21.15.37:0/1434613108 learned_addr learned my addr 172.21.15.37:0/1434613108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba13fff700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 0x7fba14101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1910b700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba14108810 0x7fba14101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 -- 172.21.15.37:0/1434613108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 msgr2=0x7fba14101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 0x7fba14101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 -- 172.21.15.37:0/1434613108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba14108810 msgr2=0x7fba14101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba14108810 0x7fba14101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.126+0000 7fba1890a700 1 -- 172.21.15.37:0/1434613108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14102350 con 0x7fba141035b0 2024-04-25T13:24:21.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba1890a700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9a0 tx=0x7fba0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:21.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0400cbd0 con 0x7fba141035b0 2024-04-25T13:24:21.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba0400cd70 con 0x7fba141035b0 2024-04-25T13:24:21.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04018680 con 0x7fba141035b0 2024-04-25T13:24:21.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba141a2c10 con 0x7fba141035b0 2024-04-25T13:24:21.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.127+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141a3150 con 0x7fba141035b0 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.128+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba14050f80 con 0x7fba141035b0 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.129+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fba04010430 con 0x7fba141035b0 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.130+0000 7fba11ffb700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 0x7fb9fc0a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.130+0000 7fba13fff700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 0x7fb9fc0a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.130+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fba040d0680 con 0x7fba141035b0 2024-04-25T13:24:21.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.130+0000 7fba13fff700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 0x7fb9fc0a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba08005950 tx=0x7fba0800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.134+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba0409cfb0 con 0x7fba141035b0 2024-04-25T13:24:21.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:20 smithi069 ceph-mon[32534]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:24:21.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:20 smithi044 ceph-mon[32590]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-25T13:24:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.429+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fba14063650 con 0x7fba141035b0 2024-04-25T13:24:21.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.433+0000 7fba11ffb700 1 -- 172.21.15.37:0/1434613108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fba0409c8b0 con 0x7fba141035b0 2024-04-25T13:24:21.433 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T13:24:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCFWSpmXhiqGRAAGTmxwe9/YkUEImVrnW/09Q== 2024-04-25T13:24:21.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 msgr2=0x7fb9fc0a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:21.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 0x7fb9fc0a62c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba08005950 tx=0x7fba0800a300 comp rx=0 tx=0).stop 2024-04-25T13:24:21.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 msgr2=0x7fba14102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba0400e9a0 tx=0x7fba0400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 shutdown_connections 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb9fc0a3e00 0x7fb9fc0a62c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba14108810 0x7fba14101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba141035b0 0x7fba14102f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 --2- 172.21.15.37:0/1434613108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba14105ee0 0x7fba14101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 >> 172.21.15.37:0/1434613108 conn(0x7fba140fcf60 msgr2=0x7fba14103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 shutdown_connections 2024-04-25T13:24:21.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:21.435+0000 7fba1ab6e700 1 -- 172.21.15.37:0/1434613108 wait complete. 2024-04-25T13:24:22.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:21 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:24:22.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:21 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:24:22.094 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:24:22.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T13:24:22.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:24:22.133 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:24:22.134 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:24:22.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T13:24:22.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:21 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:24:22.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:21 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:24:22.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:21 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T13:24:22.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:21 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1434613108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T13:24:22.420 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:23.554 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:23 smithi037 ceph-mon[29204]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T13:24:23.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:23 smithi069 ceph-mon[32534]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T13:24:23.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:23 smithi044 ceph-mon[32590]: pgmap v297: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.8 KiB/s wr, 8 op/s 2024-04-25T13:24:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 -- 172.21.15.37:0/3216564477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c107990 msgr2=0x7fe46c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 --2- 172.21.15.37:0/3216564477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c107990 0x7fe46c107d70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe460009990 tx=0x7fe460009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 -- 172.21.15.37:0/3216564477 shutdown_connections 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 --2- 172.21.15.37:0/3216564477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c102300 0x7fe46c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 --2- 172.21.15.37:0/3216564477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c107990 0x7fe46c107d70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 --2- 172.21.15.37:0/3216564477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c101940 0x7fe46c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 -- 172.21.15.37:0/3216564477 >> 172.21.15.37:0/3216564477 conn(0x7fe46c0788f0 msgr2=0x7fe46c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 -- 172.21.15.37:0/3216564477 shutdown_connections 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.072+0000 7fe470f48700 1 -- 172.21.15.37:0/3216564477 wait complete. 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 Processor -- start 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 -- start start 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 0x7fe46c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 0x7fe46c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:24.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c111b00 con 0x7fe46c102300 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c111980 con 0x7fe46c107990 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.073+0000 7fe470f48700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c111c80 con 0x7fe46c101940 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 0x7fe46c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 0x7fe46c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46a59c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 0x7fe46c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.37:56852/0 (socket says 172.21.15.37:56852) 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46ad9d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:34248/0 (socket says 172.21.15.37:34248) 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46a59c700 1 -- 172.21.15.37:0/2377846430 learned_addr learned my addr 172.21.15.37:0/2377846430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 -- 172.21.15.37:0/2377846430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 msgr2=0x7fe46c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 0x7fe46c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 -- 172.21.15.37:0/2377846430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 msgr2=0x7fe46c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 -- 172.21.15.37:0/2377846430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c1a7690 con 0x7fe46c102300 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe46ad9d700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe469d9b700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 0x7fe46c103a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00c9c0 con 0x7fe46c102300 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe45c00cb60 con 0x7fe46c102300 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a7980 con 0x7fe46c102300 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c018680 con 0x7fe46c102300 2024-04-25T13:24:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.074+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a7f10 con 0x7fe46c102300 2024-04-25T13:24:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.076+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c005330 con 0x7fe46c102300 2024-04-25T13:24:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.077+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe45c018820 con 0x7fe46c102300 2024-04-25T13:24:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.078+0000 7fe45b7fe700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.078+0000 7fe46a59c700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.078+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fe45c0d07e0 con 0x7fe46c102300 2024-04-25T13:24:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.078+0000 7fe46a59c700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 0x7fe4540a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b2e0 tx=0x7fe460009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.082+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe45c09d110 con 0x7fe46c102300 2024-04-25T13:24:24.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.353+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe44c0058f0 con 0x7fe46c102300 2024-04-25T13:24:24.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.358+0000 7fe45b7fe700 1 -- 172.21.15.37:0/2377846430 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7fe45c09ca10 con 0x7fe46c102300 2024-04-25T13:24:24.361 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 msgr2=0x7fe4540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 0x7fe4540a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe46000b2e0 tx=0x7fe460009510 comp rx=0 tx=0).stop 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 msgr2=0x7fe46c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 0x7fe46c103a30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9f0 tx=0x7fe45c00ed00 comp rx=0 tx=0).stop 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 shutdown_connections 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe4540a3d50 0x7fe4540a6210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe46c107990 0x7fe46c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe46c101940 0x7fe46c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 --2- 172.21.15.37:0/2377846430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe46c102300 0x7fe46c103a30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:24.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 >> 172.21.15.37:0/2377846430 conn(0x7fe46c0788f0 msgr2=0x7fe46c06a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:24.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 shutdown_connections 2024-04-25T13:24:24.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:24.361+0000 7fe470f48700 1 -- 172.21.15.37:0/2377846430 wait complete. 2024-04-25T13:24:24.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:24 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2377846430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:24:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:24 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2377846430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:24:24.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:24 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2377846430' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T13:24:25.006 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2819711067},{"type":"v1","addr":"172.21.15.37:6801","nonce":2819711067}]},"active_addr":"172.21.15.37:6801/2819711067","active_change":"2024-04-25T13:17:05.642939+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":4004466629}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":774754895}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":683134189}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1479124970}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2890616640}]}]} 2024-04-25T13:24:25.012 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T13:24:25.012 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T13:24:25.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:24:25.263 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:25.696 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:25 smithi037 ceph-mon[29204]: pgmap v298: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T13:24:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:25 smithi069 ceph-mon[32534]: pgmap v298: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T13:24:25.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:25 smithi044 ceph-mon[32590]: pgmap v298: 97 pgs: 97 active+clean; 615 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.886+0000 7f6916f65700 1 -- 172.21.15.37:0/1577705560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 msgr2=0x7f691010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.886+0000 7f6916f65700 1 --2- 172.21.15.37:0/1577705560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f691010f8e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f690800b0a0 tx=0x7f690800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- 172.21.15.37:0/1577705560 shutdown_connections 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- 172.21.15.37:0/1577705560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f6910108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- 172.21.15.37:0/1577705560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f691010f8e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- 172.21.15.37:0/1577705560 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 0x7f6910102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- 172.21.15.37:0/1577705560 >> 172.21.15.37:0/1577705560 conn(0x7f6910079bf0 msgr2=0x7f691007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- 172.21.15.37:0/1577705560 shutdown_connections 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- 172.21.15.37:0/1577705560 wait complete. 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 Processor -- start 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- start start 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 0x7f69101990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f691019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6910112c10 con 0x7f6910103450 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6910112a90 con 0x7f6910108a80 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.887+0000 7f6916f65700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6910112d90 con 0x7f6910102a90 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:26.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42568/0 (socket says 172.21.15.37:42568) 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 -- 172.21.15.37:0/3609999004 learned_addr learned my addr 172.21.15.37:0/3609999004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f6915502700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f691019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f6914d01700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 0x7f69101990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 -- 172.21.15.37:0/3609999004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 msgr2=0x7f69101990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 0x7f69101990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 -- 172.21.15.37:0/3609999004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 msgr2=0x7f691019d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f691019d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 -- 172.21.15.37:0/3609999004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691019e080 con 0x7f6910103450 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f6915502700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f691019d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690ffff700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00c9b0 tx=0x7f68fc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc007b00 con 0x7f6910103450 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691019e370 con 0x7f6910103450 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68fc00ce50 con 0x7f6910103450 2024-04-25T13:24:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.888+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc018660 con 0x7f6910103450 2024-04-25T13:24:26.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.889+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69101a5bf0 con 0x7f6910103450 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.889+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910069730 con 0x7f6910103450 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.891+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f68fc00f450 con 0x7f6910103450 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.891+0000 7f690dffb700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 0x7f69000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.891+0000 7f6914d01700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 0x7f69000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.891+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f68fc0d0580 con 0x7f6910103450 2024-04-25T13:24:26.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.892+0000 7f6914d01700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 0x7f69000a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6904009be0 tx=0x7f6904009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:26.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:26.895+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68fc014030 con 0x7f6910103450 2024-04-25T13:24:27.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.102+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f691019a400 con 0x7f6910103450 2024-04-25T13:24:27.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.103+0000 7f690dffb700 1 -- 172.21.15.37:0/3609999004 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22136 (secure 0 0 0) 0x7f691019a400 con 0x7f6910103450 2024-04-25T13:24:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:27.104 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:24:12.861871+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:27.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 msgr2=0x7f69000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 0x7f69000a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6904009be0 tx=0x7f6904009260 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 msgr2=0x7f6910199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00c9b0 tx=0x7f68fc00ccc0 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 shutdown_connections 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f69000a3cd0 0x7f69000a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6910108a80 0x7f691019d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6910102a90 0x7f69101990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 --2- 172.21.15.37:0/3609999004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910103450 0x7f6910199600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 >> 172.21.15.37:0/3609999004 conn(0x7f6910079bf0 msgr2=0x7f6910101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 shutdown_connections 2024-04-25T13:24:27.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:27.106+0000 7f6916f65700 1 -- 172.21.15.37:0/3609999004 wait complete. 2024-04-25T13:24:27.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:27 smithi037 ceph-mon[29204]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-04-25T13:24:27.699 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:27 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3609999004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:27.730 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T13:24:27.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T13:24:27.733 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:27 smithi069 ceph-mon[32534]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-04-25T13:24:27.734 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:27 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3609999004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:27 smithi044 ceph-mon[32590]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 6.1 KiB/s wr, 8 op/s 2024-04-25T13:24:27.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:27 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3609999004' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:27.986 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:29.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/4152905406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac108a60 msgr2=0x7f7bac108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:29.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/4152905406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac108a60 0x7f7bac108e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009990 tx=0x7f7ba8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/4152905406 shutdown_connections 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/4152905406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac06bbb0 0x7f7bac10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/4152905406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac108a60 0x7f7bac108e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/4152905406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06b1f0 0x7f7bac06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/4152905406 >> 172.21.15.37:0/4152905406 conn(0x7f7bac0789e0 msgr2=0x7f7bac078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/4152905406 shutdown_connections 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.572+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/4152905406 wait complete. 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 Processor -- start 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 -- start start 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 0x7f7bac1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac108a60 0x7f7bac19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:29.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112c60 con 0x7f7bac06b1f0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112ae0 con 0x7f7bac06bbb0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb3e9f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac112de0 con 0x7f7bac108a60 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb1c3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb1c3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42592/0 (socket says 172.21.15.37:42592) 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.573+0000 7f7bb1c3b700 1 -- 172.21.15.37:0/646999669 learned_addr learned my addr 172.21.15.37:0/646999669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb143a700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 0x7f7bac1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb243c700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac108a60 0x7f7bac19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 -- 172.21.15.37:0/646999669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac108a60 msgr2=0x7f7bac19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac108a60 0x7f7bac19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 -- 172.21.15.37:0/646999669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 msgr2=0x7f7bac1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 0x7f7bac1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 -- 172.21.15.37:0/646999669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac19e1a0 con 0x7f7bac06b1f0 2024-04-25T13:24:29.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb143a700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 0x7f7bac1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb1c3b700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8027070 con 0x7f7bac06b1f0 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac19e430 con 0x7f7bac06b1f0 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba8003fa0 con 0x7f7bac06b1f0 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8022db0 con 0x7f7bac06b1f0 2024-04-25T13:24:29.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.574+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1a5d10 con 0x7f7bac06b1f0 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.575+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac10b060 con 0x7f7bac06b1f0 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.576+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7ba802a070 con 0x7f7bac06b1f0 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.577+0000 7f7ba2ffd700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 0x7f7b980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.577+0000 7f7bb143a700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 0x7f7b980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.578+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f7ba80d10c0 con 0x7f7bac06b1f0 2024-04-25T13:24:29.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.578+0000 7f7bb143a700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 0x7f7b980a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c005950 tx=0x7f7b9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:29.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.582+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ba8013070 con 0x7f7bac06b1f0 2024-04-25T13:24:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.790+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bac19a4f0 con 0x7f7bac06b1f0 2024-04-25T13:24:29.791 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:29 smithi037 ceph-mon[29204]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-04-25T13:24:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.792+0000 7f7ba2ffd700 1 -- 172.21.15.37:0/646999669 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22136 (secure 0 0 0) 0x7f7ba809db80 con 0x7f7bac06b1f0 2024-04-25T13:24:29.792 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:24:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:24:12.861871+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 msgr2=0x7f7b980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 0x7f7b980a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c005950 tx=0x7f7b9c0058e0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 msgr2=0x7f7bac199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8000c00 tx=0x7f7ba80047b0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 shutdown_connections 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7b980a3cd0 0x7f7b980a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bac108a60 0x7f7bac19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bac06b1f0 0x7f7bac199190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 --2- 172.21.15.37:0/646999669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bac06bbb0 0x7f7bac1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 >> 172.21.15.37:0/646999669 conn(0x7f7bac0789e0 msgr2=0x7f7bac0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 shutdown_connections 2024-04-25T13:24:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:29.794+0000 7f7bb3e9f700 1 -- 172.21.15.37:0/646999669 wait complete. 2024-04-25T13:24:29.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:29 smithi069 ceph-mon[32534]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-04-25T13:24:29.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:29 smithi044 ceph-mon[32590]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-04-25T13:24:30.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T13:24:30.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T13:24:30.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T13:24:30.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T13:24:30.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T13:24:30.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T13:24:30.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T13:24:30.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T13:24:30.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T13:24:30.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T13:24:30.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T13:24:30.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T13:24:30.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:30 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/646999669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:30.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:30 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/646999669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:30.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:30 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/646999669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:24:31.319 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:31.321 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:31.321 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:31.841 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:31 smithi037 ceph-mon[29204]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-25T13:24:31.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:31 smithi069 ceph-mon[32534]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-25T13:24:31.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:31 smithi044 ceph-mon[32590]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-25T13:24:32.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.809 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:32.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:33.719 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:33 smithi037 ceph-mon[29204]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:24:33.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:33 smithi069 ceph-mon[32534]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:24:33.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:33 smithi044 ceph-mon[32590]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-25T13:24:35.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:35 smithi037 ceph-mon[29204]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T13:24:35.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:35 smithi069 ceph-mon[32534]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T13:24:35.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:35 smithi044 ceph-mon[32590]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-04-25T13:24:37.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:37 smithi037 ceph-mon[29204]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-04-25T13:24:37.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:37 smithi069 ceph-mon[32534]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-04-25T13:24:37.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:37 smithi044 ceph-mon[32590]: pgmap v304: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s 2024-04-25T13:24:39.685 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:39 smithi037 ceph-mon[29204]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-25T13:24:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.702+0000 7f954cc68700 1 -- 172.21.15.37:0/2491144774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f954806bbb0 msgr2=0x7f954810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.702+0000 7f954cc68700 1 --2- 172.21.15.37:0/2491144774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f954806bbb0 0x7f954810f930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f954000b0a0 tx=0x7f954000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 -- 172.21.15.37:0/2491144774 shutdown_connections 2024-04-25T13:24:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 --2- 172.21.15.37:0/2491144774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f954806bbb0 0x7f954810f930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 --2- 172.21.15.37:0/2491144774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9548108a60 0x7f9548108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 --2- 172.21.15.37:0/2491144774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 0x7f954806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 -- 172.21.15.37:0/2491144774 >> 172.21.15.37:0/2491144774 conn(0x7f95480789e0 msgr2=0x7f9548078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:39.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 -- 172.21.15.37:0/2491144774 shutdown_connections 2024-04-25T13:24:39.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 -- 172.21.15.37:0/2491144774 wait complete. 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 Processor -- start 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.703+0000 7f954cc68700 1 -- start start 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 0x7f95481019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f954806bbb0 0x7f9548101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 0x7f9548102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9548112d00 con 0x7f9548108a60 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9548112b80 con 0x7f954806b1f0 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954cc68700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9548112e80 con 0x7f954806bbb0 2024-04-25T13:24:39.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 0x7f9548102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 0x7f9548102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58280/0 (socket says 172.21.15.37:58280) 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 -- 172.21.15.37:0/2702959079 learned_addr learned my addr 172.21.15.37:0/2702959079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954659c700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 0x7f95481019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9545d9b700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f954806bbb0 0x7f9548101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 -- 172.21.15.37:0/2702959079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f954806bbb0 msgr2=0x7f9548101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f954806bbb0 0x7f9548101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 -- 172.21.15.37:0/2702959079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 msgr2=0x7f95481019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 0x7f95481019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95481a7770 con 0x7f9548108a60 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9545d9b700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f954806bbb0 0x7f9548101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f954659c700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f954806b1f0 0x7f95481019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.704+0000 7f9546d9d700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 0x7f9548102430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95400062a0 tx=0x7f9540004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:39.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.705+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540021070 con 0x7f9548108a60 2024-04-25T13:24:39.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.705+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95481a7a00 con 0x7f9548108a60 2024-04-25T13:24:39.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.705+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9540007d00 con 0x7f9548108a60 2024-04-25T13:24:39.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.705+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954001d450 con 0x7f9548108a60 2024-04-25T13:24:39.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.705+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a7f00 con 0x7f9548108a60 2024-04-25T13:24:39.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.706+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9548063110 con 0x7f9548108a60 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.707+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9540004000 con 0x7f9548108a60 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.707+0000 7f95377fe700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95300a3cd0 0x7f95300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f954659c700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95300a3cd0 0x7f95300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f954000f020 con 0x7f9548108a60 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f95377fe700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] conn(0x7f95300ad4b0 0x7f95300af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9540003de0 con 0x7f95300ad4b0 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f95400d2190 con 0x7f9548108a60 2024-04-25T13:24:39.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f9545d9b700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] conn(0x7f95300ad4b0 0x7f95300af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:39.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f954659c700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95300a3cd0 0x7f95300a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95380099d0 tx=0x7f9538008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:39.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.708+0000 7f9545d9b700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] conn(0x7f95300ad4b0 0x7f95300af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:39.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.709+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== osd.0 v2:172.21.15.37:6802/3385916087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9540003de0 con 0x7f95300ad4b0 2024-04-25T13:24:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.725+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 --> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9548063510 con 0x7f95300ad4b0 2024-04-25T13:24:39.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f95377fe700 1 -- 172.21.15.37:0/2702959079 <== osd.0 v2:172.21.15.37:6802/3385916087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9548063510 con 0x7f95300ad4b0 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] conn(0x7f95300ad4b0 msgr2=0x7f95300af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] conn(0x7f95300ad4b0 0x7f95300af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95300a3cd0 msgr2=0x7f95300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95300a3cd0 0x7f95300a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f95380099d0 tx=0x7f9538008040 comp rx=0 tx=0).stop 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 msgr2=0x7f9548102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 --2- 172.21.15.37:0/2702959079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9548108a60 0x7f9548102430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f95400062a0 tx=0x7f9540004ed0 comp rx=0 tx=0).stop 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f9546d9d700 1 -- 172.21.15.37:0/2702959079 reap_dead start 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 shutdown_connections 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 >> 172.21.15.37:0/2702959079 conn(0x7f95480789e0 msgr2=0x7f95480ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 shutdown_connections 2024-04-25T13:24:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:39.726+0000 7f954cc68700 1 -- 172.21.15.37:0/2702959079 wait complete. 2024-04-25T13:24:39.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:39 smithi069 ceph-mon[32534]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-25T13:24:39.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:39 smithi044 ceph-mon[32590]: pgmap v305: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-25T13:24:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 -- 172.21.15.37:0/2065466134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4814108a20 msgr2=0x7f4814108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 --2- 172.21.15.37:0/2065466134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4814108a20 0x7f4814108e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f480400b0a0 tx=0x7f480400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 -- 172.21.15.37:0/2065466134 shutdown_connections 2024-04-25T13:24:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 --2- 172.21.15.37:0/2065466134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f481406b980 0x7f481410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 --2- 172.21.15.37:0/2065466134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4814108a20 0x7f4814108e00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 --2- 172.21.15.37:0/2065466134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 0x7f481406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 -- 172.21.15.37:0/2065466134 >> 172.21.15.37:0/2065466134 conn(0x7f48140788b0 msgr2=0x7f4814078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 -- 172.21.15.37:0/2065466134 shutdown_connections 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.564+0000 7f4819377700 1 -- 172.21.15.37:0/2065466134 wait complete. 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 Processor -- start 2024-04-25T13:24:40.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 -- start start 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 0x7f4814101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f481406b980 0x7f4814101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4814108a20 0x7f4814102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4814112c20 con 0x7f481406b980 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4814112aa0 con 0x7f4814108a20 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4819377700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4814112da0 con 0x7f481406afe0 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f48127fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f481406b980 0x7f4814101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f48127fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f481406b980 0x7f4814101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58290/0 (socket says 172.21.15.37:58290) 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f48127fc700 1 -- 172.21.15.37:0/756646816 learned_addr learned my addr 172.21.15.37:0/756646816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:40.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f4812ffd700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 0x7f4814101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.565+0000 7f48137fe700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4814108a20 0x7f4814102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 -- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4814108a20 msgr2=0x7f4814102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4814108a20 0x7f4814102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 -- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f481406b980 msgr2=0x7f4814101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f481406b980 0x7f4814101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48141a7650 con 0x7f481406afe0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4812ffd700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 0x7f4814101a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f480400b070 tx=0x7f4804004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804021070 con 0x7f481406afe0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48141a78e0 con 0x7f481406afe0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4804007c00 con 0x7f481406afe0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040052a0 con 0x7f481406afe0 2024-04-25T13:24:40.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.566+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141a7d90 con 0x7f481406afe0 2024-04-25T13:24:40.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.567+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48140630d0 con 0x7f481406afe0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.569+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4804024070 con 0x7f481406afe0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.569+0000 7f480bfff700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f48000a3cd0 0x7f48000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.569+0000 7f48127fc700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f48000a3cd0 0x7f48000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.569+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f480400f020 con 0x7f481406afe0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.569+0000 7f480bfff700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] conn(0x7f48000ad380 0x7f48000af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.570+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f480402fe50 con 0x7f48000ad380 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.570+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f48040d2090 con 0x7f481406afe0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.570+0000 7f48127fc700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f48000a3cd0 0x7f48000a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4814078f20 tx=0x7f47fc009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.570+0000 7f48137fe700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] conn(0x7f48000ad380 0x7f48000af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:40.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.570+0000 7f48137fe700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] conn(0x7f48000ad380 0x7f48000af7a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:40.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.571+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== osd.4 v2:172.21.15.44:6808/844548522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f480402fe50 con 0x7f48000ad380 2024-04-25T13:24:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.588+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 --> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4814063550 con 0x7f48000ad380 2024-04-25T13:24:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.588+0000 7f480bfff700 1 -- 172.21.15.37:0/756646816 <== osd.4 v2:172.21.15.44:6808/844548522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4814063550 con 0x7f48000ad380 2024-04-25T13:24:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] conn(0x7f48000ad380 msgr2=0x7f48000af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] conn(0x7f48000ad380 0x7f48000af7a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f48000a3cd0 msgr2=0x7f48000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f48000a3cd0 0x7f48000a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4814078f20 tx=0x7f47fc009290 comp rx=0 tx=0).stop 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 msgr2=0x7f4814101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 --2- 172.21.15.37:0/756646816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f481406afe0 0x7f4814101a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f480400b070 tx=0x7f4804004e60 comp rx=0 tx=0).stop 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f48137fe700 1 -- 172.21.15.37:0/756646816 reap_dead start 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 shutdown_connections 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 >> 172.21.15.37:0/756646816 conn(0x7f48140788b0 msgr2=0x7f48140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 shutdown_connections 2024-04-25T13:24:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:40.589+0000 7f4819377700 1 -- 172.21.15.37:0/756646816 wait complete. 2024-04-25T13:24:41.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:41 smithi037 ceph-mon[29204]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:41.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:41 smithi069 ceph-mon[32534]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:41.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:41 smithi044 ceph-mon[32590]: pgmap v306: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:42.705 INFO:teuthology.orchestra.run.smithi037.stdout:150323855415 2024-04-25T13:24:42.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T13:24:43.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:43 smithi037 ceph-mon[29204]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:43.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:43 smithi069 ceph-mon[32534]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:43.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:43 smithi044 ceph-mon[32590]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-25T13:24:44.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 -- 172.21.15.37:0/3948114643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 msgr2=0x7f1cfc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/3948114643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc108e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009990 tx=0x7f1cec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 -- 172.21.15.37:0/3948114643 shutdown_connections 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/3948114643 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cfc06b980 0x7f1cfc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/3948114643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc108e00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/3948114643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cfc06afe0 0x7f1cfc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 -- 172.21.15.37:0/3948114643 >> 172.21.15.37:0/3948114643 conn(0x7f1cfc0788b0 msgr2=0x7f1cfc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:44.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 -- 172.21.15.37:0/3948114643 shutdown_connections 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.206+0000 7f1d01dc3700 1 -- 172.21.15.37:0/3948114643 wait complete. 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 Processor -- start 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 -- start start 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cfc06afe0 0x7f1cfc101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cfc06b980 0x7f1cfc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc112bc0 con 0x7f1cfc108a20 2024-04-25T13:24:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc112a40 con 0x7f1cfc06b980 2024-04-25T13:24:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.207+0000 7f1d01dc3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc112d40 con 0x7f1cfc06afe0 2024-04-25T13:24:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58298/0 (socket says 172.21.15.37:58298) 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 -- 172.21.15.37:0/1712099458 learned_addr learned my addr 172.21.15.37:0/1712099458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfaffd700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cfc06b980 0x7f1cfc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfb7fe700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cfc06afe0 0x7f1cfc101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 -- 172.21.15.37:0/1712099458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cfc06afe0 msgr2=0x7f1cfc101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cfc06afe0 0x7f1cfc101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 -- 172.21.15.37:0/1712099458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cfc06b980 msgr2=0x7f1cfc101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cfc06b980 0x7f1cfc101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc1a7660 con 0x7f1cfc108a20 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cfbfff700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc102480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007ee0 tx=0x7f1cf400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4004bb0 con 0x7f1cfc108a20 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf4004d50 con 0x7f1cfc108a20 2024-04-25T13:24:44.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc1a7950 con 0x7f1cfc108a20 2024-04-25T13:24:44.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4005920 con 0x7f1cfc108a20 2024-04-25T13:24:44.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.208+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1a7e90 con 0x7f1cfc108a20 2024-04-25T13:24:44.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.209+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1cfc063110 con 0x7f1cfc108a20 2024-04-25T13:24:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.210+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1cf401a020 con 0x7f1cfc108a20 2024-04-25T13:24:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cf8ff9700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:44.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cfb7fe700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f1cf40d00a0 con 0x7f1cfc108a20 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cf8ff9700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cf401de50 con 0x7f1ce40ad4b0 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1cf40d0580 con 0x7f1cfc108a20 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.211+0000 7f1cfaffd700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:44.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.212+0000 7f1cfb7fe700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00b2e0 tx=0x7f1cec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.212+0000 7f1cfaffd700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:44.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.213+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== osd.6 v2:172.21.15.37:6818/4082263223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cf401de50 con 0x7f1ce40ad4b0 2024-04-25T13:24:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.229+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 --> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1cfc063590 con 0x7f1ce40ad4b0 2024-04-25T13:24:44.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.229+0000 7f1cf8ff9700 1 -- 172.21.15.37:0/1712099458 <== osd.6 v2:172.21.15.37:6818/4082263223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1cfc063590 con 0x7f1ce40ad4b0 2024-04-25T13:24:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] conn(0x7f1ce40ad4b0 msgr2=0x7f1ce40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] conn(0x7f1ce40ad4b0 0x7f1ce40af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ce40a3cd0 msgr2=0x7f1ce40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1ce40a3cd0 0x7f1ce40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00b2e0 tx=0x7f1cec005960 comp rx=0 tx=0).stop 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 msgr2=0x7f1cfc102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 --2- 172.21.15.37:0/1712099458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cfc108a20 0x7f1cfc102480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4007ee0 tx=0x7f1cf400d3b0 comp rx=0 tx=0).stop 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1cfbfff700 1 -- 172.21.15.37:0/1712099458 reap_dead start 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 shutdown_connections 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 >> 172.21.15.37:0/1712099458 conn(0x7f1cfc0788b0 msgr2=0x7f1cfc0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 shutdown_connections 2024-04-25T13:24:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:44.230+0000 7f1d01dc3700 1 -- 172.21.15.37:0/1712099458 wait complete. 2024-04-25T13:24:45.545 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:45 smithi037 ceph-mon[29204]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:24:45.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:45 smithi069 ceph-mon[32534]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:24:45.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:45 smithi044 ceph-mon[32590]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T13:24:46.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 -- 172.21.15.37:0/3113716295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc101940 msgr2=0x7f51bc101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 --2- 172.21.15.37:0/3113716295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc101940 0x7f51bc101dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009990 tx=0x7f51a8009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 -- 172.21.15.37:0/3113716295 shutdown_connections 2024-04-25T13:24:46.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 --2- 172.21.15.37:0/3113716295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc107990 0x7f51bc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 --2- 172.21.15.37:0/3113716295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc102300 0x7f51bc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 --2- 172.21.15.37:0/3113716295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc101940 0x7f51bc101dc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 -- 172.21.15.37:0/3113716295 >> 172.21.15.37:0/3113716295 conn(0x7f51bc0788f0 msgr2=0x7f51bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 -- 172.21.15.37:0/3113716295 shutdown_connections 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.274+0000 7f51c30a8700 1 -- 172.21.15.37:0/3113716295 wait complete. 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 Processor -- start 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 -- start start 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc101940 0x7f51bc199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 0x7f51bc199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc107990 0x7f51bc19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc111ae0 con 0x7f51bc102300 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc111960 con 0x7f51bc101940 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51c30a8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc111c60 con 0x7f51bc107990 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 0x7f51bc199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:46.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 0x7f51bc199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58310/0 (socket says 172.21.15.37:58310) 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.275+0000 7f51bbfff700 1 -- 172.21.15.37:0/4237673070 learned_addr learned my addr 172.21.15.37:0/4237673070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51c0e44700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc101940 0x7f51bc199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51c1645700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc107990 0x7f51bc19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 -- 172.21.15.37:0/4237673070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc107990 msgr2=0x7f51bc19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51bc107990 0x7f51bc19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 -- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc101940 msgr2=0x7f51bc199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51bc101940 0x7f51bc199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc19e050 con 0x7f51bc102300 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51bbfff700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 0x7f51bc199580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51a80057f0 tx=0x7f51a8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8027070 con 0x7f51bc102300 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a80041c0 con 0x7f51bc102300 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8022e60 con 0x7f51bc102300 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.276+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc19e2e0 con 0x7f51bc102300 2024-04-25T13:24:46.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.277+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc103420 con 0x7f51bc102300 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.277+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51bc19e770 con 0x7f51bc102300 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.279+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f51a802a070 con 0x7f51bc102300 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.279+0000 7f51b9ffb700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f51ac0a3e20 0x7f51ac0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51c0e44700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f51ac0a3e20 0x7f51ac0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f51a80d1ec0 con 0x7f51bc102300 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51b9ffb700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] conn(0x7f51ac0ad600 0x7f51ac0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51a8009030 con 0x7f51ac0ad600 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f51a80d23a0 con 0x7f51bc102300 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51c0e44700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f51ac0a3e20 0x7f51ac0a62e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.280+0000 7f51c1645700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] conn(0x7f51ac0ad600 0x7f51ac0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.281+0000 7f51c1645700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] conn(0x7f51ac0ad600 0x7f51ac0afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.282+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== osd.1 v2:172.21.15.44:6800/1009231193 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51a8009030 con 0x7f51ac0ad600 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.299+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 --> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51bc0634b0 con 0x7f51ac0ad600 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51b9ffb700 1 -- 172.21.15.37:0/4237673070 <== osd.1 v2:172.21.15.44:6800/1009231193 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51bc0634b0 con 0x7f51ac0ad600 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] conn(0x7f51ac0ad600 msgr2=0x7f51ac0afa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] conn(0x7f51ac0ad600 0x7f51ac0afa20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f51ac0a3e20 msgr2=0x7f51ac0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f51ac0a3e20 0x7f51ac0a62e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005fd0 tx=0x7f51b000af20 comp rx=0 tx=0).stop 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 msgr2=0x7f51bc199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c30a8700 1 --2- 172.21.15.37:0/4237673070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51bc102300 0x7f51bc199580 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51a80057f0 tx=0x7f51a8003730 comp rx=0 tx=0).stop 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.300+0000 7f51c1645700 1 -- 172.21.15.37:0/4237673070 reap_dead start 2024-04-25T13:24:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.301+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 shutdown_connections 2024-04-25T13:24:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.301+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 >> 172.21.15.37:0/4237673070 conn(0x7f51bc0788f0 msgr2=0x7f51bc06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.301+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 shutdown_connections 2024-04-25T13:24:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.301+0000 7f51c30a8700 1 -- 172.21.15.37:0/4237673070 wait complete. 2024-04-25T13:24:47.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:47 smithi037 ceph-mon[29204]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:24:47.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:47 smithi069 ceph-mon[32534]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:24:47.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:47 smithi044 ceph-mon[32590]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 -- 172.21.15.37:0/2315697921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 msgr2=0x7fb14c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 --2- 172.21.15.37:0/2315697921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c06abb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb140009990 tx=0x7fb140009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 -- 172.21.15.37:0/2315697921 shutdown_connections 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 --2- 172.21.15.37:0/2315697921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c108a30 0x7fb14c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 --2- 172.21.15.37:0/2315697921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c06b0f0 0x7fb14c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 --2- 172.21.15.37:0/2315697921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c06abb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 -- 172.21.15.37:0/2315697921 >> 172.21.15.37:0/2315697921 conn(0x7fb14c0788f0 msgr2=0x7fb14c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 -- 172.21.15.37:0/2315697921 shutdown_connections 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.945+0000 7fb15195a700 1 -- 172.21.15.37:0/2315697921 wait complete. 2024-04-25T13:24:47.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 Processor -- start 2024-04-25T13:24:47.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 -- start start 2024-04-25T13:24:47.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c06b0f0 0x7fb14c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c108a30 0x7fb14c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112c30 con 0x7fb14c06a750 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112ab0 con 0x7fb14c06b0f0 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb15195a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112db0 con 0x7fb14c108a30 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb14affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb14affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58350/0 (socket says 172.21.15.37:58350) 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb14affd700 1 -- 172.21.15.37:0/2989114825 learned_addr learned my addr 172.21.15.37:0/2989114825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.946+0000 7fb14a7fc700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c06b0f0 0x7fb14c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14b7fe700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c108a30 0x7fb14c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 -- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c108a30 msgr2=0x7fb14c102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c108a30 0x7fb14c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 -- 172.21.15.37:0/2989114825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c06b0f0 msgr2=0x7fb14c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c06b0f0 0x7fb14c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a7690 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14b7fe700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb14c108a30 0x7fb14c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14a7fc700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c06b0f0 0x7fb14c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb14affd700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c101a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b530 tx=0x7fb13c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c015020 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c011ce0 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a7980 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c010d20 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.947+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a7ec0 con 0x7fb14c06a750 2024-04-25T13:24:47.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.948+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb14c063110 con 0x7fb14c06a750 2024-04-25T13:24:47.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.949+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb13c018020 con 0x7fb14c06a750 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb150958700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb1340a3cd0 0x7fb1340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb14a7fc700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb1340a3cd0 0x7fb1340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb13c0d0210 con 0x7fb14c06a750 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb150958700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] conn(0x7fb1340ad4b0 0x7fb1340af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb13c018ec0 con 0x7fb1340ad4b0 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb13c0d06f0 con 0x7fb14c06a750 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb14b7fe700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] conn(0x7fb1340ad4b0 0x7fb1340af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.950+0000 7fb14a7fc700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb1340a3cd0 0x7fb1340a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb14001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.951+0000 7fb14b7fe700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] conn(0x7fb1340ad4b0 0x7fb1340af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:47.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.951+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== osd.8 v2:172.21.15.69:6816/1610697891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb13c018ec0 con 0x7fb1340ad4b0 2024-04-25T13:24:47.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.968+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 --> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb14c063590 con 0x7fb1340ad4b0 2024-04-25T13:24:47.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb150958700 1 -- 172.21.15.37:0/2989114825 <== osd.8 v2:172.21.15.69:6816/1610697891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb14c063590 con 0x7fb1340ad4b0 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] conn(0x7fb1340ad4b0 msgr2=0x7fb1340af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] conn(0x7fb1340ad4b0 0x7fb1340af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb1340a3cd0 msgr2=0x7fb1340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb1340a3cd0 0x7fb1340a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb14001a040 comp rx=0 tx=0).stop 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 msgr2=0x7fb14c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 --2- 172.21.15.37:0/2989114825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb14c06a750 0x7fb14c101a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b530 tx=0x7fb13c00b8f0 comp rx=0 tx=0).stop 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb14b7fe700 1 -- 172.21.15.37:0/2989114825 reap_dead start 2024-04-25T13:24:47.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 shutdown_connections 2024-04-25T13:24:47.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.969+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 >> 172.21.15.37:0/2989114825 conn(0x7fb14c0788f0 msgr2=0x7fb14c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:47.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.970+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 shutdown_connections 2024-04-25T13:24:47.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:47.970+0000 7fb15195a700 1 -- 172.21.15.37:0/2989114825 wait complete. 2024-04-25T13:24:48.596 INFO:teuthology.orchestra.run.smithi037.stdout:206158430254 2024-04-25T13:24:48.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T13:24:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:34359738443 2024-04-25T13:24:49.184 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 -- 172.21.15.37:0/178379005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c107990 msgr2=0x7f1b4c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 --2- 172.21.15.37:0/178379005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c107990 0x7f1b4c107d70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009990 tx=0x7f1b3c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 -- 172.21.15.37:0/178379005 shutdown_connections 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 --2- 172.21.15.37:0/178379005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4c102300 0x7f1b4c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 --2- 172.21.15.37:0/178379005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c107990 0x7f1b4c107d70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 --2- 172.21.15.37:0/178379005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c101940 0x7f1b4c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 -- 172.21.15.37:0/178379005 >> 172.21.15.37:0/178379005 conn(0x7f1b4c0788f0 msgr2=0x7f1b4c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 -- 172.21.15.37:0/178379005 shutdown_connections 2024-04-25T13:24:49.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.355+0000 7f1b52198700 1 -- 172.21.15.37:0/178379005 wait complete. 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 Processor -- start 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 -- start start 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 0x7f1b4c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c102300 0x7f1b4c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4c107990 0x7f1b4c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c111b70 con 0x7f1b4c101940 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c1119f0 con 0x7f1b4c102300 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b52198700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c111cf0 con 0x7f1b4c107990 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b4b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 0x7f1b4c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b4b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 0x7f1b4c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58328/0 (socket says 172.21.15.37:58328) 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.356+0000 7f1b4b7fe700 1 -- 172.21.15.37:0/2221880649 learned_addr learned my addr 172.21.15.37:0/2221880649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4affd700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c102300 0x7f1b4c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:49.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 -- 172.21.15.37:0/2221880649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4c107990 msgr2=0x7f1b4c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b4c107990 0x7f1b4c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 -- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c102300 msgr2=0x7f1b4c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c102300 0x7f1b4c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c19e1a0 con 0x7f1b4c101940 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4affd700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b4c102300 0x7f1b4c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b4b7fe700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 0x7f1b4c1991e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:49.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c027070 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b3c0041d0 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c00f460 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c19e430 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.357+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1a5d10 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.358+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b4c06aa70 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.359+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b3c02a070 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b48ff9700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1b340a3d50 0x7f1b340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b4affd700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1b340a3d50 0x7f1b340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f1b3c0d1b40 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b48ff9700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] conn(0x7f1b340ad530 0x7f1b340af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b3c019e80 con 0x7f1b340ad530 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.360+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f1b3c0d1ef0 con 0x7f1b4c101940 2024-04-25T13:24:49.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.361+0000 7f1b4affd700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1b340a3d50 0x7f1b340a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b40005fd0 tx=0x7f1b40005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.361+0000 7f1b4bfff700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] conn(0x7f1b340ad530 0x7f1b340af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.361+0000 7f1b4bfff700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] conn(0x7f1b340ad530 0x7f1b340af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.362+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== osd.10 v2:172.21.15.44:6824/2564518662 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b3c019e80 con 0x7f1b340ad530 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.378+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 --> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b4c103060 con 0x7f1b340ad530 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b48ff9700 1 -- 172.21.15.37:0/2221880649 <== osd.10 v2:172.21.15.44:6824/2564518662 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b4c103060 con 0x7f1b340ad530 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] conn(0x7f1b340ad530 msgr2=0x7f1b340af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] conn(0x7f1b340ad530 0x7f1b340af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1b340a3d50 msgr2=0x7f1b340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f1b340a3d50 0x7f1b340a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b40005fd0 tx=0x7f1b40005dc0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 msgr2=0x7f1b4c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 --2- 172.21.15.37:0/2221880649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4c101940 0x7f1b4c1991e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c0047b0 comp rx=0 tx=0).stop 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b4bfff700 1 -- 172.21.15.37:0/2221880649 reap_dead start 2024-04-25T13:24:49.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 shutdown_connections 2024-04-25T13:24:49.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 >> 172.21.15.37:0/2221880649 conn(0x7f1b4c0788f0 msgr2=0x7f1b4c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:49.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 shutdown_connections 2024-04-25T13:24:49.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:46.379+0000 7f1b52198700 1 -- 172.21.15.37:0/2221880649 wait complete. 2024-04-25T13:24:49.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:49 smithi037 ceph-mon[29204]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:49.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:49 smithi069 ceph-mon[32534]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:49.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:49 smithi044 ceph-mon[32590]: pgmap v310: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:49.938 INFO:teuthology.orchestra.run.smithi037.stdout:55834574919 2024-04-25T13:24:49.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T13:24:51.442 INFO:teuthology.orchestra.run.smithi037.stdout:326417514522 2024-04-25T13:24:51.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T13:24:51.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:51 smithi037 ceph-mon[29204]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:51.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:51 smithi069 ceph-mon[32534]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:51.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:51 smithi044 ceph-mon[32590]: pgmap v311: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T13:24:52.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 -- 172.21.15.37:0/256880129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820101970 msgr2=0x7f9820101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 --2- 172.21.15.37:0/256880129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820101970 0x7f9820101df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9808009990 tx=0x7f9808009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 -- 172.21.15.37:0/256880129 shutdown_connections 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 --2- 172.21.15.37:0/256880129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820107920 0x7f9820107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 --2- 172.21.15.37:0/256880129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9820102330 0x7f982010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 --2- 172.21.15.37:0/256880129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820101970 0x7f9820101df0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 -- 172.21.15.37:0/256880129 >> 172.21.15.37:0/256880129 conn(0x7f98200779b0 msgr2=0x7f9820077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 -- 172.21.15.37:0/256880129 shutdown_connections 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.656+0000 7f982553a700 1 -- 172.21.15.37:0/256880129 wait complete. 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.657+0000 7f982553a700 1 Processor -- start 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.657+0000 7f982553a700 1 -- start start 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820101970 0x7f9820198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9820102330 0x7f9820198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 0x7f982019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:52.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9820111b00 con 0x7f9820107920 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9820111980 con 0x7f9820101970 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f982553a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9820111c80 con 0x7f9820102330 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 0x7f982019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 0x7f982019c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52658/0 (socket says 172.21.15.37:52658) 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 -- 172.21.15.37:0/177153405 learned_addr learned my addr 172.21.15.37:0/177153405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981e7fc700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9820102330 0x7f9820198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981effd700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820101970 0x7f9820198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 -- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9820102330 msgr2=0x7f9820198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9820102330 0x7f9820198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 -- 172.21.15.37:0/177153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820101970 msgr2=0x7f9820198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820101970 0x7f9820198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982019d040 con 0x7f9820107920 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981effd700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9820101970 0x7f9820198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:52.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.658+0000 7f981f7fe700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 0x7f982019c900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f981800c900 tx=0x7f981800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.659+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818007b00 con 0x7f9820107920 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.659+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981800ce50 con 0x7f9820107920 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.659+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982019d330 con 0x7f9820107920 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.659+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818018660 con 0x7f9820107920 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.659+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982006b1b0 con 0x7f9820107920 2024-04-25T13:24:52.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.660+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f982006b7a0 con 0x7f9820107920 2024-04-25T13:24:52.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.662+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f981800f450 con 0x7f9820107920 2024-04-25T13:24:52.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f9817fff700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f980c0a3cd0 0x7f980c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:52.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f98180d07f0 con 0x7f9820107920 2024-04-25T13:24:52.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f981effd700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f980c0a3cd0 0x7f980c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f9817fff700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] conn(0x7f980c0ad4b0 0x7f980c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f980c0aff80 con 0x7f980c0ad4b0 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f98180d0ba0 con 0x7f9820107920 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f981effd700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f980c0a3cd0 0x7f980c0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9810009790 tx=0x7f9810009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.663+0000 7f981e7fc700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] conn(0x7f980c0ad4b0 0x7f980c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:52.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.664+0000 7f981e7fc700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] conn(0x7f980c0ad4b0 0x7f980c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:52.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.664+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== osd.2 v2:172.21.15.69:6800/3261521434 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f980c0aff80 con 0x7f980c0ad4b0 2024-04-25T13:24:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.700+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 --> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f982006bc20 con 0x7f980c0ad4b0 2024-04-25T13:24:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f9817fff700 1 -- 172.21.15.37:0/177153405 <== osd.2 v2:172.21.15.69:6800/3261521434 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f982006bc20 con 0x7f980c0ad4b0 2024-04-25T13:24:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] conn(0x7f980c0ad4b0 msgr2=0x7f980c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] conn(0x7f980c0ad4b0 0x7f980c0af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f980c0a3cd0 msgr2=0x7f980c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f980c0a3cd0 0x7f980c0a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9810009790 tx=0x7f9810009040 comp rx=0 tx=0).stop 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 msgr2=0x7f982019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 --2- 172.21.15.37:0/177153405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9820107920 0x7f982019c900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f981800c900 tx=0x7f981800cc10 comp rx=0 tx=0).stop 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f981f7fe700 1 -- 172.21.15.37:0/177153405 reap_dead start 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 shutdown_connections 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.701+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 >> 172.21.15.37:0/177153405 conn(0x7f98200779b0 msgr2=0x7f98200ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.702+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 shutdown_connections 2024-04-25T13:24:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:52.702+0000 7f982553a700 1 -- 172.21.15.37:0/177153405 wait complete. 2024-04-25T13:24:53.520 INFO:teuthology.orchestra.run.smithi037.stdout:77309411396 2024-04-25T13:24:53.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T13:24:53.742 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:53 smithi037 ceph-mon[29204]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:53.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:53 smithi069 ceph-mon[32534]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:53.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:53 smithi044 ceph-mon[32590]: pgmap v312: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 -- 172.21.15.37:0/3231979973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c107990 msgr2=0x7f219c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 --2- 172.21.15.37:0/3231979973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c107990 0x7f219c107d70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2190009990 tx=0x7f2190009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 -- 172.21.15.37:0/3231979973 shutdown_connections 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 --2- 172.21.15.37:0/3231979973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f219c102300 0x7f219c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 --2- 172.21.15.37:0/3231979973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c107990 0x7f219c107d70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 --2- 172.21.15.37:0/3231979973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c101940 0x7f219c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 -- 172.21.15.37:0/3231979973 >> 172.21.15.37:0/3231979973 conn(0x7f219c0788f0 msgr2=0x7f219c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 -- 172.21.15.37:0/3231979973 shutdown_connections 2024-04-25T13:24:54.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.308+0000 7f21a1f19700 1 -- 172.21.15.37:0/3231979973 wait complete. 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.309+0000 7f21a1f19700 1 Processor -- start 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.309+0000 7f21a1f19700 1 -- start start 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.309+0000 7f21a1f19700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 0x7f219c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.309+0000 7f21a1f19700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f219c102300 0x7f219c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.309+0000 7f21a1f19700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c107990 0x7f219c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f21a1f19700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f219c111bd0 con 0x7f219c101940 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f21a1f19700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f219c111a50 con 0x7f219c107990 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f21a1f19700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f219c111d50 con 0x7f219c102300 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 0x7f219c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 0x7f219c1034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52694/0 (socket says 172.21.15.37:52694) 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 -- 172.21.15.37:0/1559562017 learned_addr learned my addr 172.21.15.37:0/1559562017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219bfff700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c107990 0x7f219c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219affd700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f219c102300 0x7f219c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 -- 172.21.15.37:0/1559562017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f219c102300 msgr2=0x7f219c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f219c102300 0x7f219c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 -- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c107990 msgr2=0x7f219c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c107990 0x7f219c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c1a7790 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219bfff700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f219c107990 0x7f219c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f219b7fe700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 0x7f219c1034f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f21900057f0 tx=0x7f2190003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190027070 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.310+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c1a7a20 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.311+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c1a7f50 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.311+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21900041c0 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.311+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190022e60 con 0x7f219c101940 2024-04-25T13:24:54.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.311+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2188001060 con 0x7f219c101940 2024-04-25T13:24:54.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.313+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f219002a070 con 0x7f219c101940 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f2198ff9700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21840a3cd0 0x7f21840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f21900d1fa0 con 0x7f219c101940 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f2198ff9700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] conn(0x7f21840ad4b0 0x7f21840af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2190009030 con 0x7f21840ad4b0 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f21900d2350 con 0x7f219c101940 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.314+0000 7f219affd700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21840a3cd0 0x7f21840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.315+0000 7f219bfff700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] conn(0x7f21840ad4b0 0x7f21840af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.315+0000 7f219affd700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21840a3cd0 0x7f21840a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f218c006fd0 tx=0x7f218c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.315+0000 7f219bfff700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] conn(0x7f21840ad4b0 0x7f21840af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.316+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== osd.7 v2:172.21.15.44:6816/3060345546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2190009030 con 0x7f21840ad4b0 2024-04-25T13:24:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.333+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 --> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2188002e90 con 0x7f21840ad4b0 2024-04-25T13:24:54.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.333+0000 7f2198ff9700 1 -- 172.21.15.37:0/1559562017 <== osd.7 v2:172.21.15.44:6816/3060345546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2188002e90 con 0x7f21840ad4b0 2024-04-25T13:24:54.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] conn(0x7f21840ad4b0 msgr2=0x7f21840af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] conn(0x7f21840ad4b0 0x7f21840af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21840a3cd0 msgr2=0x7f21840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21840a3cd0 0x7f21840a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f218c006fd0 tx=0x7f218c008040 comp rx=0 tx=0).stop 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 msgr2=0x7f219c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 --2- 172.21.15.37:0/1559562017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f219c101940 0x7f219c1034f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f21900057f0 tx=0x7f2190003730 comp rx=0 tx=0).stop 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f219bfff700 1 -- 172.21.15.37:0/1559562017 reap_dead start 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 shutdown_connections 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 >> 172.21.15.37:0/1559562017 conn(0x7f219c0788f0 msgr2=0x7f219c06a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 shutdown_connections 2024-04-25T13:24:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.334+0000 7f21a1f19700 1 -- 172.21.15.37:0/1559562017 wait complete. 2024-04-25T13:24:54.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 -- 172.21.15.37:0/1870093111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c806a750 msgr2=0x7f83c806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 --2- 172.21.15.37:0/1870093111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c806a750 0x7f83c806abb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009990 tx=0x7f83bc009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 -- 172.21.15.37:0/1870093111 shutdown_connections 2024-04-25T13:24:54.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 --2- 172.21.15.37:0/1870093111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c8108a30 0x7f83c8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 --2- 172.21.15.37:0/1870093111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806b0f0 0x7f83c810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 --2- 172.21.15.37:0/1870093111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c806a750 0x7f83c806abb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 -- 172.21.15.37:0/1870093111 >> 172.21.15.37:0/1870093111 conn(0x7f83c80788f0 msgr2=0x7f83c8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 -- 172.21.15.37:0/1870093111 shutdown_connections 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.961+0000 7f83ccde7700 1 -- 172.21.15.37:0/1870093111 wait complete. 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.962+0000 7f83ccde7700 1 Processor -- start 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.962+0000 7f83ccde7700 1 -- start start 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.962+0000 7f83ccde7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806a750 0x7f83c8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.962+0000 7f83ccde7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c806b0f0 0x7f83c8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 0x7f83c8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8112c10 con 0x7f83c8108a30 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8112a90 con 0x7f83c806b0f0 2024-04-25T13:24:54.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83c8112d90 con 0x7f83c806a750 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 0x7f83c8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c806b0f0 0x7f83c8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c5d9b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c806b0f0 0x7f83c8101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:56562/0 (socket says 172.21.15.37:56562) 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 0x7f83c8102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52678/0 (socket says 172.21.15.37:52678) 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c5d9b700 1 -- 172.21.15.37:0/2444341274 learned_addr learned my addr 172.21.15.37:0/2444341274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c659c700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806a750 0x7f83c8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 -- 172.21.15.37:0/2444341274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806a750 msgr2=0x7f83c8101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806a750 0x7f83c8101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 -- 172.21.15.37:0/2444341274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c806b0f0 msgr2=0x7f83c8101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83c806b0f0 0x7f83c8101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c81a7690 con 0x7f83c8108a30 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c659c700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83c806a750 0x7f83c8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:54.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83c6d9d700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 0x7f83c8102490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83c000efc0 tx=0x7f83c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00092e0 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81a7920 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83c0004bb0 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004040 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.963+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81a7e60 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.964+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83c8063110 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.965+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f83c001b030 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83b77fe700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f83b00a3cd0 0x7f83b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83c659c700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f83b00a3cd0 0x7f83b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f83c00d08e0 con 0x7f83c8108a30 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83b77fe700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] conn(0x7f83b00ad4b0 0x7f83b00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83c001de50 con 0x7f83b00ad4b0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.966+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f83c00d0dc0 con 0x7f83c8108a30 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.967+0000 7f83c659c700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f83b00a3cd0 0x7f83b00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005e10 tx=0x7f83b8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.967+0000 7f83c5d9b700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] conn(0x7f83b00ad4b0 0x7f83b00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.967+0000 7f83c5d9b700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] conn(0x7f83b00ad4b0 0x7f83b00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.968+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== osd.3 v2:172.21.15.37:6810/1255089911 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83c001de50 con 0x7f83b00ad4b0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 --> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83c8063510 con 0x7f83b00ad4b0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83b77fe700 1 -- 172.21.15.37:0/2444341274 <== osd.3 v2:172.21.15.37:6810/1255089911 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f83c8063510 con 0x7f83b00ad4b0 2024-04-25T13:24:54.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] conn(0x7f83b00ad4b0 msgr2=0x7f83b00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] conn(0x7f83b00ad4b0 0x7f83b00af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f83b00a3cd0 msgr2=0x7f83b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f83b00a3cd0 0x7f83b00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005e10 tx=0x7f83b8005d60 comp rx=0 tx=0).stop 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 msgr2=0x7f83c8102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.984+0000 7f83ccde7700 1 --2- 172.21.15.37:0/2444341274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c8108a30 0x7f83c8102490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83c000efc0 tx=0x7f83c000c3a0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.985+0000 7f83c6d9d700 1 -- 172.21.15.37:0/2444341274 reap_dead start 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.985+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 shutdown_connections 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.985+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 >> 172.21.15.37:0/2444341274 conn(0x7f83c80788f0 msgr2=0x7f83c80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.985+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 shutdown_connections 2024-04-25T13:24:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:53.985+0000 7f83ccde7700 1 -- 172.21.15.37:0/2444341274 wait complete. 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 -- 172.21.15.37:0/2065307378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102300 msgr2=0x7f322c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 --2- 172.21.15.37:0/2065307378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102300 0x7f322c10e7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3224009990 tx=0x7f3224009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 -- 172.21.15.37:0/2065307378 shutdown_connections 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 --2- 172.21.15.37:0/2065307378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c102300 0x7f322c10e7f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 --2- 172.21.15.37:0/2065307378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c107990 0x7f322c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 --2- 172.21.15.37:0/2065307378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c101940 0x7f322c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 -- 172.21.15.37:0/2065307378 >> 172.21.15.37:0/2065307378 conn(0x7f322c0788f0 msgr2=0x7f322c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 -- 172.21.15.37:0/2065307378 shutdown_connections 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.951+0000 7f32316f9700 1 -- 172.21.15.37:0/2065307378 wait complete. 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 Processor -- start 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 -- start start 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 0x7f322c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c102300 0x7f322c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c107990 0x7f322c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f322c111bb0 con 0x7f322c101940 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f322c111a30 con 0x7f322c107990 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f32316f9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f322c111d30 con 0x7f322c102300 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f322affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 0x7f322c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f322affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 0x7f322c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52632/0 (socket says 172.21.15.37:52632) 2024-04-25T13:24:54.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.952+0000 7f322affd700 1 -- 172.21.15.37:0/2736449592 learned_addr learned my addr 172.21.15.37:0/2736449592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322b7fe700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c107990 0x7f322c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322a7fc700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c102300 0x7f322c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 -- 172.21.15.37:0/2736449592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c102300 msgr2=0x7f322c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c102300 0x7f322c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 -- 172.21.15.37:0/2736449592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c107990 msgr2=0x7f322c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f322c107990 0x7f322c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322c19cff0 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322a7fc700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f322c102300 0x7f322c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f322affd700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 0x7f322c197fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ba50 tx=0x7f321c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c019070 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322c19d2e0 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f321c014440 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c00ce60 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.953+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322c1a4b60 con 0x7f322c101940 2024-04-25T13:24:54.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.954+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f322c063190 con 0x7f322c101940 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.955+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f321c01c070 con 0x7f322c101940 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f3213fff700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32140a3cd0 0x7f32140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f322a7fc700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32140a3cd0 0x7f32140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f321c0cfff0 con 0x7f322c101940 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f3213fff700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] conn(0x7f32140ad4b0 0x7f32140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f321c01ce50 con 0x7f32140ad4b0 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f321c0d03a0 con 0x7f322c101940 2024-04-25T13:24:54.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f322a7fc700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32140a3cd0 0x7f32140a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f3220005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.956+0000 7f322b7fe700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] conn(0x7f32140ad4b0 0x7f32140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.957+0000 7f322b7fe700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] conn(0x7f32140ad4b0 0x7f32140af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.957+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== osd.9 v2:172.21.15.37:6826/682284085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f321c01ce50 con 0x7f32140ad4b0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 --> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f322c063610 con 0x7f32140ad4b0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f3213fff700 1 -- 172.21.15.37:0/2736449592 <== osd.9 v2:172.21.15.37:6826/682284085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f322c063610 con 0x7f32140ad4b0 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] conn(0x7f32140ad4b0 msgr2=0x7f32140af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f32316f9700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] conn(0x7f32140ad4b0 0x7f32140af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32140a3cd0 msgr2=0x7f32140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.974+0000 7f32316f9700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f32140a3cd0 0x7f32140a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f3220005e20 comp rx=0 tx=0).stop 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 msgr2=0x7f322c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 --2- 172.21.15.37:0/2736449592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f322c101940 0x7f322c197fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ba50 tx=0x7f321c00bd60 comp rx=0 tx=0).stop 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f322b7fe700 1 -- 172.21.15.37:0/2736449592 reap_dead start 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 shutdown_connections 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 >> 172.21.15.37:0/2736449592 conn(0x7f322c0788f0 msgr2=0x7f322c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:54.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 shutdown_connections 2024-04-25T13:24:54.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:50.975+0000 7f32316f9700 1 -- 172.21.15.37:0/2736449592 wait complete. 2024-04-25T13:24:55.494 INFO:teuthology.orchestra.run.smithi037.stdout:115964117054 2024-04-25T13:24:55.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T13:24:55.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:55 smithi037 ceph-mon[29204]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:55.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:55 smithi069 ceph-mon[32534]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:55.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:55 smithi044 ceph-mon[32590]: pgmap v313: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 -- 172.21.15.37:0/914106027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 msgr2=0x7fa9a0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 --2- 172.21.15.37:0/914106027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0101df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa994009990 tx=0x7fa994009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 -- 172.21.15.37:0/914106027 shutdown_connections 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 --2- 172.21.15.37:0/914106027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0107920 0x7fa9a0107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 --2- 172.21.15.37:0/914106027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a0102330 0x7fa9a010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 --2- 172.21.15.37:0/914106027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0101df0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.782+0000 7fa9a515e700 1 -- 172.21.15.37:0/914106027 >> 172.21.15.37:0/914106027 conn(0x7fa9a00788f0 msgr2=0x7fa9a0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.783+0000 7fa9a515e700 1 -- 172.21.15.37:0/914106027 shutdown_connections 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.783+0000 7fa9a515e700 1 -- 172.21.15.37:0/914106027 wait complete. 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.783+0000 7fa9a515e700 1 Processor -- start 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.783+0000 7fa9a515e700 1 -- start start 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0102330 0x7fa9a0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a0107920 0x7fa9a019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0111bb0 con 0x7fa9a0101970 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0111a30 con 0x7fa9a0102330 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa9a515e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a0111d30 con 0x7fa9a0107920 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52712/0 (socket says 172.21.15.37:52712) 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 -- 172.21.15.37:0/3688683081 learned_addr learned my addr 172.21.15.37:0/3688683081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99e59c700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0102330 0x7fa9a0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99f59e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a0107920 0x7fa9a019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 -- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a0107920 msgr2=0x7fa9a019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa9a0107920 0x7fa9a019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 -- 172.21.15.37:0/3688683081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0102330 msgr2=0x7fa9a0199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9a0102330 0x7fa9a0199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a019e110 con 0x7fa9a0101970 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa99ed9d700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0199100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b530 tx=0x7fa99000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.784+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990015020 con 0x7fa9a0101970 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.785+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa990011ce0 con 0x7fa9a0101970 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.785+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a019e400 con 0x7fa9a0101970 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.785+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990010d20 con 0x7fa9a0101970 2024-04-25T13:24:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.785+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a01a5c80 con 0x7fa9a0101970 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.785+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9a00631d0 con 0x7fa9a0101970 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa990018020 con 0x7fa9a0101970 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa987fff700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa9880a3cd0 0x7fa9880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa99e59c700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa9880a3cd0 0x7fa9880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa9900d0260 con 0x7fa9a0101970 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa987fff700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] conn(0x7fa9880ad4b0 0x7fa9880af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.787+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa990010ec0 con 0x7fa9880ad4b0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.788+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa9900d0740 con 0x7fa9a0101970 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.788+0000 7fa99e59c700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa9880a3cd0 0x7fa9880a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b5c0 tx=0x7fa99401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.788+0000 7fa99f59e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] conn(0x7fa9880ad4b0 0x7fa9880af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.788+0000 7fa99f59e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] conn(0x7fa9880ad4b0 0x7fa9880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.789+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== osd.11 v2:172.21.15.69:6824/2045769419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa990010ec0 con 0x7fa9880ad4b0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.805+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 --> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9a0063650 con 0x7fa9880ad4b0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa987fff700 1 -- 172.21.15.37:0/3688683081 <== osd.11 v2:172.21.15.69:6824/2045769419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9a0063650 con 0x7fa9880ad4b0 2024-04-25T13:24:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] conn(0x7fa9880ad4b0 msgr2=0x7fa9880af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] conn(0x7fa9880ad4b0 0x7fa9880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa9880a3cd0 msgr2=0x7fa9880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa9880a3cd0 0x7fa9880a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa99400b5c0 tx=0x7fa99401a040 comp rx=0 tx=0).stop 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 msgr2=0x7fa9a0199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 --2- 172.21.15.37:0/3688683081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9a0101970 0x7fa9a0199100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b530 tx=0x7fa99000b8f0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa99f59e700 1 -- 172.21.15.37:0/3688683081 reap_dead start 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 shutdown_connections 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 >> 172.21.15.37:0/3688683081 conn(0x7fa9a00788f0 msgr2=0x7fa9a006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 shutdown_connections 2024-04-25T13:24:56.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:54.806+0000 7fa9a515e700 1 -- 172.21.15.37:0/3688683081 wait complete. 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 -- 172.21.15.37:0/819618961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 msgr2=0x7f35fc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 --2- 172.21.15.37:0/819618961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc101db0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009990 tx=0x7f35ec009ca0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 -- 172.21.15.37:0/819618961 shutdown_connections 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 --2- 172.21.15.37:0/819618961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 0x7f35fc10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 --2- 172.21.15.37:0/819618961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 0x7f35fc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 --2- 172.21.15.37:0/819618961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc101db0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 -- 172.21.15.37:0/819618961 >> 172.21.15.37:0/819618961 conn(0x7f35fc0788b0 msgr2=0x7f35fc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 -- 172.21.15.37:0/819618961 shutdown_connections 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.681+0000 7f3604190700 1 -- 172.21.15.37:0/819618961 wait complete. 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 Processor -- start 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 -- start start 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 0x7f35fc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 0x7f35fc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc111b60 con 0x7f35fc101930 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1119e0 con 0x7f35fc1022f0 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3604190700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc111ce0 con 0x7f35fc107980 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52720/0 (socket says 172.21.15.37:52720) 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 -- 172.21.15.37:0/3419551453 learned_addr learned my addr 172.21.15.37:0/3419551453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:24:56.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f360272d700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 0x7f35fc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f360172b700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 0x7f35fc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 -- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 msgr2=0x7f35fc19da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 0x7f35fc19da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 -- 172.21.15.37:0/3419551453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 msgr2=0x7f35fc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 0x7f35fc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f3601f2c700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19e150 con 0x7f35fc101930 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f360272d700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35fc107980 0x7f35fc19da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.682+0000 7f360172b700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35fc1022f0 0x7f35fc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f3601f2c700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc199140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35f800d980 tx=0x7f35f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8017070 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f8004830 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc19e440 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8005440 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.683+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1a5cc0 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.684+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35fc0ffaf0 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.685+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f35f80055e0 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f35f2ffd700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35e80a3dd0 0x7f35e80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f360172b700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35e80a3dd0 0x7f35e80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f35f80d0990 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f35f2ffd700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] conn(0x7f35e80ad5b0 0x7f35e80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35e80b0080 con 0x7f35e80ad5b0 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f35f80d0d80 con 0x7f35fc101930 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f360172b700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35e80a3dd0 0x7f35e80a6290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00b5c0 tx=0x7f35ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.686+0000 7f360272d700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] conn(0x7f35e80ad5b0 0x7f35e80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:24:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.687+0000 7f360272d700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] conn(0x7f35e80ad5b0 0x7f35e80af9d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.688+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== osd.5 v2:172.21.15.69:6808/3958240806 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35e80b0080 con 0x7f35e80ad5b0 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.704+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 --> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35fc103050 con 0x7f35e80ad5b0 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f35f2ffd700 1 -- 172.21.15.37:0/3419551453 <== osd.5 v2:172.21.15.69:6808/3958240806 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f35fc103050 con 0x7f35e80ad5b0 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] conn(0x7f35e80ad5b0 msgr2=0x7f35e80af9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] conn(0x7f35e80ad5b0 0x7f35e80af9d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35e80a3dd0 msgr2=0x7f35e80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f35e80a3dd0 0x7f35e80a6290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00b5c0 tx=0x7f35ec005fb0 comp rx=0 tx=0).stop 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 msgr2=0x7f35fc199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 --2- 172.21.15.37:0/3419551453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35fc101930 0x7f35fc199140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35f800d980 tx=0x7f35f800dd40 comp rx=0 tx=0).stop 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f360272d700 1 -- 172.21.15.37:0/3419551453 reap_dead start 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 shutdown_connections 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 >> 172.21.15.37:0/3419551453 conn(0x7f35fc0788b0 msgr2=0x7f35fc105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 shutdown_connections 2024-04-25T13:24:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:24:55.705+0000 7f3604190700 1 -- 172.21.15.37:0/3419551453 wait complete. 2024-04-25T13:24:57.345 INFO:teuthology.orchestra.run.smithi037.stdout:360777252886 2024-04-25T13:24:57.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T13:24:57.477 INFO:teuthology.orchestra.run.smithi037.stdout:270582939685 2024-04-25T13:24:57.477 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T13:24:57.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:57 smithi037 ceph-mon[29204]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:57.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:57 smithi069 ceph-mon[32534]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:57.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:57 smithi044 ceph-mon[32590]: pgmap v314: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:24:57.908 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:57.913 INFO:teuthology.orchestra.run.smithi037.stdout:231928234027 2024-04-25T13:24:57.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T13:24:57.968 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:57.991 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:58.271 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:24:59.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:24:59 smithi037 ceph-mon[29204]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:59.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:24:59 smithi069 ceph-mon[32534]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:24:59.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:24:59 smithi044 ceph-mon[32590]: pgmap v315: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:00.528 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:00.560 INFO:teuthology.orchestra.run.smithi037.stdout:171798691893 2024-04-25T13:25:00.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T13:25:01.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:01 smithi037 ceph-mon[29204]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:25:01.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:01 smithi069 ceph-mon[32534]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:25:01.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:01 smithi044 ceph-mon[32590]: pgmap v316: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-25T13:25:03.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:03 smithi037 ceph-mon[29204]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:03.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:03 smithi069 ceph-mon[32534]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:03.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:03 smithi044 ceph-mon[32590]: pgmap v317: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:05.772 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:05 smithi037 ceph-mon[29204]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:05.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:05 smithi069 ceph-mon[32534]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:05.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:05 smithi044 ceph-mon[32590]: pgmap v318: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:06.419 INFO:teuthology.orchestra.run.smithi037.stdout:304942678048 2024-04-25T13:25:06.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T13:25:06.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-25T13:25:06.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:25:06.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:06 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:25:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T13:25:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:06 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T13:25:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:06 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-25T13:25:07.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:07.675 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:25:07.676 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:07 smithi037 ceph-mon[29204]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:25:07.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:07.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T13:25:07.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:25:07.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:25:07.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:07 smithi069 ceph-mon[32534]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:25:07.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:07.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-25T13:25:07.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-25T13:25:07.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:07 smithi044 ceph-mon[32590]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 -- 172.21.15.37:0/452856156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 msgr2=0x7fe10410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 --2- 172.21.15.37:0/452856156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe10410f8f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b0a0 tx=0x7fe0f400b3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 -- 172.21.15.37:0/452856156 shutdown_connections 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 --2- 172.21.15.37:0/452856156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe104108a20 0x7fe104108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 --2- 172.21.15.37:0/452856156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe10410f8f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 --2- 172.21.15.37:0/452856156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe10406afe0 0x7fe10406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 -- 172.21.15.37:0/452856156 >> 172.21.15.37:0/452856156 conn(0x7fe1040788b0 msgr2=0x7fe104078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:08.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 -- 172.21.15.37:0/452856156 shutdown_connections 2024-04-25T13:25:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.015+0000 7fe1096a3700 1 -- 172.21.15.37:0/452856156 wait complete. 2024-04-25T13:25:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 Processor -- start 2024-04-25T13:25:08.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 -- start start 2024-04-25T13:25:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10406afe0 0x7fe104101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:08.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104108a20 0x7fe104102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112ca0 con 0x7fe10406b980 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112b20 con 0x7fe10406afe0 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1096a3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe104112e20 con 0x7fe104108a20 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41418/0 (socket says 172.21.15.37:41418) 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1027fc700 1 -- 172.21.15.37:0/2900661332 learned_addr learned my addr 172.21.15.37:0/2900661332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe102ffd700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10406afe0 0x7fe104101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.016+0000 7fe1037fe700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104108a20 0x7fe104102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 -- 172.21.15.37:0/2900661332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104108a20 msgr2=0x7fe104102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104108a20 0x7fe104102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 -- 172.21.15.37:0/2900661332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10406afe0 msgr2=0x7fe104101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10406afe0 0x7fe104101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 -- 172.21.15.37:0/2900661332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1041a7760 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1027fc700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00b530 tx=0x7fe0ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec015020 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1041a7a50 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec011ce0 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec010e40 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.017+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041a7fe0 con 0x7fe10406b980 2024-04-25T13:25:08.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.018+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10410b020 con 0x7fe10406b980 2024-04-25T13:25:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.019+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0ec018020 con 0x7fe10406b980 2024-04-25T13:25:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.020+0000 7fe0fbfff700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 0x7fe0f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.020+0000 7fe102ffd700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 0x7fe0f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.020+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fe0ec0d01f0 con 0x7fe10406b980 2024-04-25T13:25:08.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.020+0000 7fe102ffd700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 0x7fe0f00a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009a10 tx=0x7fe0fc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.024+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0ec01f020 con 0x7fe10406b980 2024-04-25T13:25:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.233+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe10410aa00 con 0x7fe10406b980 2024-04-25T13:25:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.233+0000 7fe0fbfff700 1 -- 172.21.15.37:0/2900661332 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe0ec09cb60 con 0x7fe10406b980 2024-04-25T13:25:08.235 INFO:teuthology.orchestra.run.smithi037.stdout:34359738448 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 msgr2=0x7fe0f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 0x7fe0f00a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009a10 tx=0x7fe0fc009190 comp rx=0 tx=0).stop 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 msgr2=0x7fe104101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00b530 tx=0x7fe0ec00b840 comp rx=0 tx=0).stop 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 shutdown_connections 2024-04-25T13:25:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fe0f00a3cd0 0x7fe0f00a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe10406afe0 0x7fe104101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe104108a20 0x7fe104102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 --2- 172.21.15.37:0/2900661332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe10406b980 0x7fe104101f40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 >> 172.21.15.37:0/2900661332 conn(0x7fe1040788b0 msgr2=0x7fe1040ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 shutdown_connections 2024-04-25T13:25:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:08.236+0000 7fe1096a3700 1 -- 172.21.15.37:0/2900661332 wait complete. 2024-04-25T13:25:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:08 smithi037 ceph-mon[29204]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:25:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:08 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2900661332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:25:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:08 smithi069 ceph-mon[32534]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:25:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:08 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2900661332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:25:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:08 smithi044 ceph-mon[32590]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T13:25:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:08 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2900661332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T13:25:08.965 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738448 for osd.0 2024-04-25T13:25:08.966 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:09.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 -- 172.21.15.37:0/3085531811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 msgr2=0x7fac9c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 --2- 172.21.15.37:0/3085531811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c107d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac98009990 tx=0x7fac98009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 -- 172.21.15.37:0/3085531811 shutdown_connections 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 --2- 172.21.15.37:0/3085531811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac9c1022f0 0x7fac9c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 --2- 172.21.15.37:0/3085531811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c107d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 --2- 172.21.15.37:0/3085531811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c101930 0x7fac9c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 -- 172.21.15.37:0/3085531811 >> 172.21.15.37:0/3085531811 conn(0x7fac9c0788b0 msgr2=0x7fac9c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 -- 172.21.15.37:0/3085531811 shutdown_connections 2024-04-25T13:25:09.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.250+0000 7faca45d0700 1 -- 172.21.15.37:0/3085531811 wait complete. 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 Processor -- start 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 -- start start 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac9c101930 0x7fac9c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c111ae0 con 0x7fac9c107980 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c111960 con 0x7fac9c101930 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca45d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c111c60 con 0x7fac9c1022f0 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca1b6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca1b6b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.37:38172/0 (socket says 172.21.15.37:38172) 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.251+0000 7faca1b6b700 1 -- 172.21.15.37:0/1990424006 learned_addr learned my addr 172.21.15.37:0/1990424006 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca2b6d700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 -- 172.21.15.37:0/1990424006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac9c101930 msgr2=0x7fac9c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac9c101930 0x7fac9c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 -- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 msgr2=0x7fac9c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 -- 172.21.15.37:0/1990424006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c1a0330 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.252+0000 7faca1b6b700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00e9c0 tx=0x7fac8c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.253+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00cbd0 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.253+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac8c00cd70 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.253+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1a0620 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.253+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c005610 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.253+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c1a7ea0 con 0x7fac9c1022f0 2024-04-25T13:25:09.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.254+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c069730 con 0x7fac9c1022f0 2024-04-25T13:25:09.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.255+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fac8c004750 con 0x7fac9c1022f0 2024-04-25T13:25:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.256+0000 7fac937fe700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 0x7fac880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.256+0000 7faca236c700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 0x7fac880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.256+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fac8c0d0a80 con 0x7fac9c1022f0 2024-04-25T13:25:09.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.256+0000 7faca236c700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 0x7fac880a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac980098e0 tx=0x7fac98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:09.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.259+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac8c09d280 con 0x7fac9c1022f0 2024-04-25T13:25:09.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.473+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fac9c19c680 con 0x7fac9c1022f0 2024-04-25T13:25:09.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.473+0000 7fac937fe700 1 -- 172.21.15.37:0/1990424006 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fac8c09cb80 con 0x7fac9c1022f0 2024-04-25T13:25:09.475 INFO:teuthology.orchestra.run.smithi037.stdout:326417514526 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 msgr2=0x7fac880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 0x7fac880a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac980098e0 tx=0x7fac98005fb0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 msgr2=0x7fac9c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00e9c0 tx=0x7fac8c00ecd0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 shutdown_connections 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fac880a3cd0 0x7fac880a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac9c101930 0x7fac9c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac9c107980 0x7fac9c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 --2- 172.21.15.37:0/1990424006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac9c1022f0 0x7fac9c19b8b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 >> 172.21.15.37:0/1990424006 conn(0x7fac9c0788b0 msgr2=0x7fac9c06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 shutdown_connections 2024-04-25T13:25:09.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.476+0000 7faca45d0700 1 -- 172.21.15.37:0/1990424006 wait complete. 2024-04-25T13:25:09.781 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:09 smithi037 ceph-mon[29204]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:09.781 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:09 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/328229403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:25:09.781 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:09 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1990424006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:25:09.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:09 smithi069 ceph-mon[32534]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:09.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:09 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/328229403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:25:09.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:09 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1990424006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- 172.21.15.37:0/2346765333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0103480 msgr2=0x7f7db010f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2346765333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0103480 0x7f7db010f980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7da800b0a0 tx=0x7f7da800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- 172.21.15.37:0/2346765333 shutdown_connections 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2346765333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db0108ab0 0x7f7db0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2346765333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0103480 0x7f7db010f980 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2346765333 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- 172.21.15.37:0/2346765333 >> 172.21.15.37:0/2346765333 conn(0x7f7db0079bf0 msgr2=0x7f7db007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- 172.21.15.37:0/2346765333 shutdown_connections 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- 172.21.15.37:0/2346765333 wait complete. 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 Processor -- start 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.884+0000 7f7db4b44700 1 -- start start 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db0103480 0x7f7db0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0112cf0 con 0x7f7db0108ab0 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0112b70 con 0x7f7db0103480 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0112e70 con 0x7f7db0102ac0 2024-04-25T13:25:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48650/0 (socket says 172.21.15.37:48650) 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 -- 172.21.15.37:0/2723787057 learned_addr learned my addr 172.21.15.37:0/2723787057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7dae59c700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 -- 172.21.15.37:0/2723787057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 msgr2=0x7f7db01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 -- 172.21.15.37:0/2723787057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db0103480 msgr2=0x7f7db0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db0103480 0x7f7db0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 -- 172.21.15.37:0/2723787057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db019e0d0 con 0x7f7db0108ab0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7dae59c700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7daed9d700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7da800b830 tx=0x7f7da8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8021070 con 0x7f7db0108ab0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db019e360 con 0x7f7db0108ab0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da8003c00 con 0x7f7db0108ab0 2024-04-25T13:25:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8004730 con 0x7f7db0108ab0 2024-04-25T13:25:09.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.885+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01a5c40 con 0x7f7db0108ab0 2024-04-25T13:25:09.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.887+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0069730 con 0x7f7db0108ab0 2024-04-25T13:25:09.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.888+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7da8003db0 con 0x7f7db0108ab0 2024-04-25T13:25:09.892 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:09 smithi044 ceph-mon[32590]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:09.892 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/328229403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T13:25:09.892 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:09 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1990424006' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T13:25:09.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.889+0000 7f7da77fe700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 0x7f7d9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:09.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.889+0000 7f7dae59c700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 0x7f7d9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:09.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.889+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f7da800f020 con 0x7f7db0108ab0 2024-04-25T13:25:09.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.889+0000 7f7dae59c700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 0x7f7d9c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7da0005fd0 tx=0x7f7da0005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:09.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.893+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7da802a020 con 0x7f7db0108ab0 2024-04-25T13:25:10.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.107+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7db019a420 con 0x7f7db0108ab0 2024-04-25T13:25:10.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.108+0000 7f7da77fe700 1 -- 172.21.15.37:0/2723787057 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7da802e3e0 con 0x7f7db0108ab0 2024-04-25T13:25:10.109 INFO:teuthology.orchestra.run.smithi037.stdout:270582939689 2024-04-25T13:25:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 msgr2=0x7f7d9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 0x7f7d9c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7da0005fd0 tx=0x7f7da0005d40 comp rx=0 tx=0).stop 2024-04-25T13:25:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 msgr2=0x7f7db019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7da800b830 tx=0x7f7da8004d90 comp rx=0 tx=0).stop 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 shutdown_connections 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f7d9c0a3d50 0x7f7d9c0a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7db0108ab0 0x7f7db019d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0102ac0 0x7f7db01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 --2- 172.21.15.37:0/2723787057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7db0103480 0x7f7db0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 >> 172.21.15.37:0/2723787057 conn(0x7f7db0079bf0 msgr2=0x7f7db0100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 shutdown_connections 2024-04-25T13:25:10.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:10.110+0000 7f7db4b44700 1 -- 172.21.15.37:0/2723787057 wait complete. 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 -- 172.21.15.37:0/2520694787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 msgr2=0x7fa8a8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/2520694787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a8108e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009990 tx=0x7fa89c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 -- 172.21.15.37:0/2520694787 shutdown_connections 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/2520694787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806b980 0x7fa8a810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/2520694787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a8108e00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/2520694787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806afe0 0x7fa8a806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 -- 172.21.15.37:0/2520694787 >> 172.21.15.37:0/2520694787 conn(0x7fa8a80788b0 msgr2=0x7fa8a8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:10.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 -- 172.21.15.37:0/2520694787 shutdown_connections 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.156+0000 7fa8aeef0700 1 -- 172.21.15.37:0/2520694787 wait complete. 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 Processor -- start 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 -- start start 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806afe0 0x7fa8a8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806b980 0x7fa8a8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8112bc0 con 0x7fa8a8108a20 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8112a40 con 0x7fa8a806b980 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8aeef0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8112d40 con 0x7fa8a806afe0 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8ad48d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8ad48d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41442/0 (socket says 172.21.15.37:41442) 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8ad48d700 1 -- 172.21.15.37:0/328229403 learned_addr learned my addr 172.21.15.37:0/328229403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8a7fff700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806b980 0x7fa8a8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8acc8c700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806afe0 0x7fa8a8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:10.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.157+0000 7fa8ad48d700 1 -- 172.21.15.37:0/328229403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806afe0 msgr2=0x7fa8a8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8ad48d700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806afe0 0x7fa8a8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8ad48d700 1 -- 172.21.15.37:0/328229403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806b980 msgr2=0x7fa8a8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8ad48d700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806b980 0x7fa8a8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8ad48d700 1 -- 172.21.15.37:0/328229403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a819e120 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8ad48d700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ef30 tx=0x7fa8a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00092e0 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8a0004bb0 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004280 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a819e3b0 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.158+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a5c90 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.159+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8069730 con 0x7fa8a8108a20 2024-04-25T13:25:10.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.160+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa8a001b030 con 0x7fa8a8108a20 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stdout:206158430259 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.161+0000 7fa8a5ffb700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 0x7fa8980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.161+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fa8a00d0050 con 0x7fa8a8108a20 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.161+0000 7fa8acc8c700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 0x7fa8980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.162+0000 7fa8acc8c700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 0x7fa8980a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.166+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8a0022020 con 0x7fa8a8108a20 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.386+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa8a802df80 con 0x7fa8a8108a20 2024-04-25T13:25:10.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.386+0000 7fa8a5ffb700 1 -- 172.21.15.37:0/328229403 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa8a0010040 con 0x7fa8a8108a20 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.388+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 msgr2=0x7fa8980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.388+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 0x7fa8980a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa89c006010 tx=0x7fa89c01a040 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.388+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 msgr2=0x7fa8a819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.388+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000ef30 tx=0x7fa8a000c3a0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 shutdown_connections 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fa8980a3cd0 0x7fa8980a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8a8108a20 0x7fa8a819d9e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8a806afe0 0x7fa8a8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 --2- 172.21.15.37:0/328229403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8a806b980 0x7fa8a8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 >> 172.21.15.37:0/328229403 conn(0x7fa8a80788b0 msgr2=0x7fa8a80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 shutdown_connections 2024-04-25T13:25:10.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:09.389+0000 7fa8aeef0700 1 -- 172.21.15.37:0/328229403 wait complete. 2024-04-25T13:25:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:10 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2723787057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:25:10.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:10 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2723787057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:25:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:10 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2723787057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T13:25:11.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939685 got 270582939689 for osd.8 2024-04-25T13:25:11.300 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:11.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:11 smithi037 ceph-mon[29204]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:25:11.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:11 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:11.820 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:11 smithi044 ceph-mon[32590]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:25:11.820 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:11 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:11.875 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:11 smithi069 ceph-mon[32534]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 637 KiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-25T13:25:11.875 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:11 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2283220561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0106930 msgr2=0x7f74a0106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2283220561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0106930 0x7f74a0106d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7490009990 tx=0x7f7490009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2283220561 shutdown_connections 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2283220561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2283220561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0106930 0x7f74a0106d10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2283220561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0101c90 0x7f74a0102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2283220561 >> 172.21.15.37:0/2283220561 conn(0x7f74a00777b0 msgr2=0x7f74a0077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2283220561 shutdown_connections 2024-04-25T13:25:12.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.195+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2283220561 wait complete. 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 Processor -- start 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 -- start start 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 0x7f74a01033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a0103930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0106930 0x7f74a0103e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0111b50 con 0x7f74a0101c90 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01119d0 con 0x7f74a0106930 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f74a5f4f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0111cd0 con 0x7f74a0102650 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 0x7f74a01033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.196+0000 7f749f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 0x7f74a01033f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48680/0 (socket says 172.21.15.37:48680) 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749f7fe700 1 -- 172.21.15.37:0/2272585325 learned_addr learned my addr 172.21.15.37:0/2272585325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a0103930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749ffff700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0106930 0x7f74a0103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 -- 172.21.15.37:0/2272585325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0106930 msgr2=0x7f74a0103e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:12.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0106930 0x7f74a0103e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 -- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 msgr2=0x7f74a01033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 0x7f74a01033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 -- 172.21.15.37:0/2272585325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01a7650 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749effd700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a0103930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f748800e9f0 tx=0x7f748800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748800c9c0 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01a7940 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f748800cb60 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488018680 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.197+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a7ed0 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.199+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a01030b0 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.200+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7488010430 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.200+0000 7f749cff9700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 0x7f748c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.200+0000 7f749f7fe700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 0x7f748c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.200+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f74880d0720 con 0x7f74a0102650 2024-04-25T13:25:12.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.201+0000 7f749f7fe700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 0x7f748c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f749000b2e0 tx=0x7f7490005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:12.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.205+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f748809cf20 con 0x7f74a0102650 2024-04-25T13:25:12.224 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:12.224 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:12.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.456+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f74a01a80d0 con 0x7f74a0102650 2024-04-25T13:25:12.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.457+0000 7f749cff9700 1 -- 172.21.15.37:0/2272585325 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f748809c820 con 0x7f74a0102650 2024-04-25T13:25:12.458 INFO:teuthology.orchestra.run.smithi037.stdout:115964117058 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 msgr2=0x7f748c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 0x7f748c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f749000b2e0 tx=0x7f7490005fd0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 msgr2=0x7f74a0103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a0103930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f748800e9f0 tx=0x7f748800ed00 comp rx=0 tx=0).stop 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 shutdown_connections 2024-04-25T13:25:12.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f748c0a3cd0 0x7f748c0a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f74a0106930 0x7f74a0103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a0101c90 0x7f74a01033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 --2- 172.21.15.37:0/2272585325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74a0102650 0x7f74a0103930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 >> 172.21.15.37:0/2272585325 conn(0x7f74a00777b0 msgr2=0x7f74a01055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 shutdown_connections 2024-04-25T13:25:12.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.459+0000 7f74a5f4f700 1 -- 172.21.15.37:0/2272585325 wait complete. 2024-04-25T13:25:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:12 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2272585325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:25:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:12 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:12 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2272585325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:25:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:12 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:12 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2272585325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T13:25:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:12 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:13.223 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117054 got 115964117058 for osd.3 2024-04-25T13:25:13.223 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:13.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:13 smithi037 ceph-mon[29204]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 0 op/s; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:13.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:13 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1349247177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:25:13.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430254 got 206158430259 for osd.6 2024-04-25T13:25:13.818 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:13.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:13 smithi069 ceph-mon[32534]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 0 op/s; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:13.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:13 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1349247177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:25:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:13 smithi044 ceph-mon[32590]: pgmap v324: 97 pgs: 1 peering, 96 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s rd, 0 op/s; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:13.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:13 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1349247177' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T13:25:13.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 -- 172.21.15.37:0/3678596648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174102ac0 msgr2=0x7ff174102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:13.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 --2- 172.21.15.37:0/3678596648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174102ac0 0x7ff174102f20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009990 tx=0x7ff15c009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 -- 172.21.15.37:0/3678596648 shutdown_connections 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 --2- 172.21.15.37:0/3678596648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174103460 0x7ff17410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 --2- 172.21.15.37:0/3678596648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174108a70 0x7ff174108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 --2- 172.21.15.37:0/3678596648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174102ac0 0x7ff174102f20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 -- 172.21.15.37:0/3678596648 >> 172.21.15.37:0/3678596648 conn(0x7ff174079bf0 msgr2=0x7ff17407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 -- 172.21.15.37:0/3678596648 shutdown_connections 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.478+0000 7ff178bde700 1 -- 172.21.15.37:0/3678596648 wait complete. 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 Processor -- start 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 -- start start 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174102ac0 0x7ff174074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 0x7ff1740757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff174112c80 con 0x7ff174103460 2024-04-25T13:25:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff174112b00 con 0x7ff174108a70 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff178bde700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff174112e00 con 0x7ff174102ac0 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.479+0000 7ff171d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48704/0 (socket says 172.21.15.37:48704) 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 -- 172.21.15.37:0/2894644920 learned_addr learned my addr 172.21.15.37:0/2894644920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff17259c700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174102ac0 0x7ff174074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff172d9d700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 0x7ff1740757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 -- 172.21.15.37:0/2894644920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174102ac0 msgr2=0x7ff174074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174102ac0 0x7ff174074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 -- 172.21.15.37:0/2894644920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 msgr2=0x7ff1740757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 0x7ff1740757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 -- 172.21.15.37:0/2894644920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1741a76e0 con 0x7ff174103460 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff172d9d700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 0x7ff1740757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:25:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff171d9b700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff15c005910 tx=0x7ff15c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c027070 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1741a7970 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff15c003f20 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c022c50 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.480+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741a7e20 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.481+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff174069730 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.483+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff15c02a070 con 0x7ff174103460 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.484+0000 7ff16b7fe700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 0x7ff1600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.484+0000 7ff17259c700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 0x7ff1600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:13.904 INFO:teuthology.orchestra.run.smithi037.stdout:55834574925 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.484+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7ff15c0d1520 con 0x7ff174103460 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.484+0000 7ff17259c700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 0x7ff1600a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005950 tx=0x7ff16c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.492+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff15c05eff0 con 0x7ff174103460 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.703+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff17402e140 con 0x7ff174103460 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.704+0000 7ff16b7fe700 1 -- 172.21.15.37:0/2894644920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff15c013070 con 0x7ff174103460 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 msgr2=0x7ff1600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 0x7ff1600a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005950 tx=0x7ff16c0058e0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 msgr2=0x7ff174075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff15c005910 tx=0x7ff15c009f90 comp rx=0 tx=0).stop 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 shutdown_connections 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff1600a3cd0 0x7ff1600a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff174108a70 0x7ff1740757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff174102ac0 0x7ff174074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 --2- 172.21.15.37:0/2894644920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff174103460 0x7ff174075280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 >> 172.21.15.37:0/2894644920 conn(0x7ff174079bf0 msgr2=0x7ff174100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 shutdown_connections 2024-04-25T13:25:13.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:13.706+0000 7ff178bde700 1 -- 172.21.15.37:0/2894644920 wait complete. 2024-04-25T13:25:14.417 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574925 for osd.1 2024-04-25T13:25:14.417 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:14.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:14 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2894644920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:25:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:14 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2894644920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:25:14.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:14 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2894644920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T13:25:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.437+0000 7fc29ea94700 1 -- 172.21.15.37:0/1704754800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 msgr2=0x7fc298102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.437+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1704754800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298102f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc284009990 tx=0x7fc284009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 -- 172.21.15.37:0/1704754800 shutdown_connections 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1704754800 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc298108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1704754800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc29810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1704754800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298102f40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 -- 172.21.15.37:0/1704754800 >> 172.21.15.37:0/1704754800 conn(0x7fc298079bf0 msgr2=0x7fc29807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 -- 172.21.15.37:0/1704754800 shutdown_connections 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 -- 172.21.15.37:0/1704754800 wait complete. 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 Processor -- start 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 -- start start 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.438+0000 7fc29ea94700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29ea94700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc29819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29ea94700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc298112d40 con 0x7fc298102ac0 2024-04-25T13:25:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29ea94700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc298112bc0 con 0x7fc298108ab0 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29ea94700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc298112ec0 con 0x7fc298103480 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc297fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc297fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.37:50230/0 (socket says 172.21.15.37:50230) 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48688/0 (socket says 172.21.15.37:48688) 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc297fff700 1 -- 172.21.15.37:0/1349247177 learned_addr learned my addr 172.21.15.37:0/1349247177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29d031700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc29819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 -- 172.21.15.37:0/1349247177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 msgr2=0x7fc298199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 -- 172.21.15.37:0/1349247177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 msgr2=0x7fc29819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc29819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 -- 172.21.15.37:0/1349247177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29819e240 con 0x7fc298102ac0 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc297fff700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29d031700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc29819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29c830700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00e9f0 tx=0x7fc28c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00c9c0 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc28c00cb60 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29819e530 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.439+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c018680 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.440+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a5db0 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.441+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298069730 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.442+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc28c010430 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.443+0000 7fc295ffb700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 0x7fc2880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.443+0000 7fc297fff700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 0x7fc2880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.443+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fc28c0d09c0 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.443+0000 7fc297fff700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 0x7fc2880a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc29807a310 tx=0x7fc28401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.447+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc28c09d220 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.679+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc29802df80 con 0x7fc298102ac0 2024-04-25T13:25:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.679+0000 7fc295ffb700 1 -- 172.21.15.37:0/1349247177 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc28c09cb20 con 0x7fc298102ac0 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 msgr2=0x7fc2880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 0x7fc2880a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc29807a310 tx=0x7fc28401a040 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 msgr2=0x7fc298199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00e9f0 tx=0x7fc28c00ed00 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 shutdown_connections 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.681+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2880a3cd0 0x7fc2880a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc298108ab0 0x7fc29819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc298102ac0 0x7fc298199230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 --2- 172.21.15.37:0/1349247177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc298103480 0x7fc298199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 >> 172.21.15.37:0/1349247177 conn(0x7fc298079bf0 msgr2=0x7fc298100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 shutdown_connections 2024-04-25T13:25:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:12.682+0000 7fc29ea94700 1 -- 172.21.15.37:0/1349247177 wait complete. 2024-04-25T13:25:14.983 INFO:teuthology.orchestra.run.smithi037.stdout:150323855421 2024-04-25T13:25:15.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855415 got 150323855421 for osd.4 2024-04-25T13:25:15.604 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 -- 172.21.15.37:0/1939634957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006b980 msgr2=0x7f5f1010f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 --2- 172.21.15.37:0/1939634957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006b980 0x7f5f1010f8f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b0a0 tx=0x7f5f0800b3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 -- 172.21.15.37:0/1939634957 shutdown_connections 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 --2- 172.21.15.37:0/1939634957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f10108a20 0x7f5f10108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 --2- 172.21.15.37:0/1939634957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006b980 0x7f5f1010f8f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 --2- 172.21.15.37:0/1939634957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f1006afe0 0x7f5f1006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 -- 172.21.15.37:0/1939634957 >> 172.21.15.37:0/1939634957 conn(0x7f5f100788b0 msgr2=0x7f5f10078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 -- 172.21.15.37:0/1939634957 shutdown_connections 2024-04-25T13:25:15.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.795+0000 7f5f15aff700 1 -- 172.21.15.37:0/1939634957 wait complete. 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 Processor -- start 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 -- start start 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f10108a20 0x7f5f1019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112be0 con 0x7f5f1006afe0 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112a60 con 0x7f5f1006b980 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.796+0000 7f5f15aff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f10112d60 con 0x7f5f10108a20 2024-04-25T13:25:15.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48726/0 (socket says 172.21.15.37:48726) 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0effd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:52180/0 (socket says 172.21.15.37:52180) 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 -- 172.21.15.37:0/195641108 learned_addr learned my addr 172.21.15.37:0/195641108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0ffff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f10108a20 0x7f5f1019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 -- 172.21.15.37:0/195641108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f10108a20 msgr2=0x7f5f1019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f10108a20 0x7f5f1019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 -- 172.21.15.37:0/195641108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 msgr2=0x7f5f101995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 -- 172.21.15.37:0/195641108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1019e0c0 con 0x7f5f1006afe0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0effd700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.797+0000 7f5f0f7fe700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d980 tx=0x7f5f0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.798+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04016070 con 0x7f5f1006afe0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.798+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f04004830 con 0x7f5f1006afe0 2024-04-25T13:25:15.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.798+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1019e3b0 con 0x7f5f1006afe0 2024-04-25T13:25:15.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.798+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04005540 con 0x7f5f1006afe0 2024-04-25T13:25:15.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.798+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f101a5c30 con 0x7f5f1006afe0 2024-04-25T13:25:15.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.799+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f10069730 con 0x7f5f1006afe0 2024-04-25T13:25:15.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.800+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f04004220 con 0x7f5f1006afe0 2024-04-25T13:25:15.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.801+0000 7f5f0cff9700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 0x7f5ef80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:15.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.801+0000 7f5f0effd700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 0x7f5ef80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:15.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.801+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f5f040d02b0 con 0x7f5f1006afe0 2024-04-25T13:25:15.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.801+0000 7f5f0effd700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 0x7f5ef80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f10078e90 tx=0x7f5f0000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:15.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:15.807+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f0401f020 con 0x7f5f1006afe0 2024-04-25T13:25:15.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:15 smithi037 ceph-mon[29204]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:15.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:15 smithi069 ceph-mon[32534]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:15.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:15 smithi044 ceph-mon[32590]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:16.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.022+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5f1002df80 con 0x7f5f1006afe0 2024-04-25T13:25:16.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.022+0000 7f5f0cff9700 1 -- 172.21.15.37:0/195641108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f04022350 con 0x7f5f1006afe0 2024-04-25T13:25:16.023 INFO:teuthology.orchestra.run.smithi037.stdout:77309411400 2024-04-25T13:25:16.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 msgr2=0x7f5ef80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 0x7f5ef80a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f10078e90 tx=0x7f5f0000b3c0 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 msgr2=0x7f5f101990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d980 tx=0x7f5f0400dc90 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 shutdown_connections 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f5ef80a3cd0 0x7f5ef80a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f10108a20 0x7f5f1019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5f1006afe0 0x7f5f101990b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 --2- 172.21.15.37:0/195641108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f1006b980 0x7f5f101995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 >> 172.21.15.37:0/195641108 conn(0x7f5f100788b0 msgr2=0x7f5f100ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 shutdown_connections 2024-04-25T13:25:16.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:16.024+0000 7f5f15aff700 1 -- 172.21.15.37:0/195641108 wait complete. 2024-04-25T13:25:16.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:16 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/195641108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:25:16.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:16 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/195641108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:25:16.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:16 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/195641108' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T13:25:17.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411400 for osd.2 2024-04-25T13:25:17.439 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:17.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:17 smithi037 ceph-mon[29204]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s rd, 0 op/s; 6.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:17.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:17 smithi069 ceph-mon[32534]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s rd, 0 op/s; 6.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:17.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:17 smithi044 ceph-mon[32590]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 111 B/s rd, 0 op/s; 6.4 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:19.221 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514522 got 326417514526 for osd.10 2024-04-25T13:25:19.221 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:19.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 -- 172.21.15.37:0/2639314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf0075590 msgr2=0x7f9cf010ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/2639314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf0075590 0x7f9cf010ef30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009990 tx=0x7f9ce0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 -- 172.21.15.37:0/2639314960 shutdown_connections 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/2639314960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0074c70 0x7f9cf0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/2639314960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf010f470 0x7f9cf0111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/2639314960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf0075590 0x7f9cf010ef30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.656+0000 7f9cf65c8700 1 -- 172.21.15.37:0/2639314960 >> 172.21.15.37:0/2639314960 conn(0x7f9cf0079bb0 msgr2=0x7f9cf0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.657+0000 7f9cf65c8700 1 -- 172.21.15.37:0/2639314960 shutdown_connections 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.657+0000 7f9cf65c8700 1 -- 172.21.15.37:0/2639314960 wait complete. 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.657+0000 7f9cf65c8700 1 Processor -- start 2024-04-25T13:25:19.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.657+0000 7f9cf65c8700 1 -- start start 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.657+0000 7f9cf65c8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf0074c70 0x7f9cf019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0075590 0x7f9cf019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0114f20 con 0x7f9cf010f470 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0114da0 con 0x7f9cf0075590 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf01150a0 con 0x7f9cf0074c70 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41340/0 (socket says 172.21.15.37:41340) 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 -- 172.21.15.37:0/839432899 learned_addr learned my addr 172.21.15.37:0/839432899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cef7fe700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0075590 0x7f9cf019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9ceffff700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf0074c70 0x7f9cf019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:19.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 -- 172.21.15.37:0/839432899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf0074c70 msgr2=0x7f9cf019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf0074c70 0x7f9cf019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 -- 172.21.15.37:0/839432899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0075590 msgr2=0x7f9cf019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0075590 0x7f9cf019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 -- 172.21.15.37:0/839432899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf01a0450 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf4b65700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800b9e0 tx=0x7f9ce800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800d8e0 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf01a0740 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce8015420 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8014630 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.658+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf01a7fc0 con 0x7f9cf010f470 2024-04-25T13:25:19.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.660+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf0069730 con 0x7f9cf010f470 2024-04-25T13:25:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.661+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ce80155d0 con 0x7f9cf010f470 2024-04-25T13:25:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.662+0000 7f9ced7fa700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 0x7f9cd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.662+0000 7f9ceffff700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 0x7f9cd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.662+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f9ce80cfea0 con 0x7f9cf010f470 2024-04-25T13:25:19.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.662+0000 7f9ceffff700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 0x7f9cd80a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009a10 tx=0x7f9ce4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:19.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.666+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ce809c700 con 0x7f9cf010f470 2024-04-25T13:25:19.694 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:19.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:19 smithi037 ceph-mon[29204]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s rd, 0 op/s; 5.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:19.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:19 smithi069 ceph-mon[32534]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s rd, 0 op/s; 5.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:19.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.887+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9cf002df80 con 0x7f9cf010f470 2024-04-25T13:25:19.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.887+0000 7f9ced7fa700 1 -- 172.21.15.37:0/839432899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9ce809c000 con 0x7f9cf010f470 2024-04-25T13:25:19.888 INFO:teuthology.orchestra.run.smithi037.stdout:360777252891 2024-04-25T13:25:19.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:19 smithi044 ceph-mon[32590]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 100 B/s rd, 0 op/s; 5.8 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.889+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 msgr2=0x7f9cd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 0x7f9cd80a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009a10 tx=0x7f9ce4009190 comp rx=0 tx=0).stop 2024-04-25T13:25:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 msgr2=0x7f9cf019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800b9e0 tx=0x7f9ce800bcf0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 shutdown_connections 2024-04-25T13:25:19.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9cd80a3cd0 0x7f9cd80a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cf010f470 0x7f9cf019fd10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cf0074c70 0x7f9cf019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 --2- 172.21.15.37:0/839432899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cf0075590 0x7f9cf019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 >> 172.21.15.37:0/839432899 conn(0x7f9cf0079bb0 msgr2=0x7f9cf00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 shutdown_connections 2024-04-25T13:25:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:19.890+0000 7f9cf65c8700 1 -- 172.21.15.37:0/839432899 wait complete. 2024-04-25T13:25:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:20 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/839432899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:25:20.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:20 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/839432899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:25:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:20 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/839432899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T13:25:21.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252886 got 360777252891 for osd.11 2024-04-25T13:25:21.336 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 -- 172.21.15.37:0/1459815532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94107990 msgr2=0x7f6e94107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/1459815532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94107990 0x7f6e94107d70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009990 tx=0x7f6e90009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 -- 172.21.15.37:0/1459815532 shutdown_connections 2024-04-25T13:25:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/1459815532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94102300 0x7f6e9410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/1459815532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94107990 0x7f6e94107d70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/1459815532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 0x7f6e94101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 -- 172.21.15.37:0/1459815532 >> 172.21.15.37:0/1459815532 conn(0x7f6e940788f0 msgr2=0x7f6e94078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 -- 172.21.15.37:0/1459815532 shutdown_connections 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.667+0000 7f6e9c24e700 1 -- 172.21.15.37:0/1459815532 wait complete. 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 Processor -- start 2024-04-25T13:25:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 -- start start 2024-04-25T13:25:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 0x7f6e941990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94107990 0x7f6e9419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94111b00 con 0x7f6e94102300 2024-04-25T13:25:21.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94111980 con 0x7f6e94101940 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9c24e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94111c80 con 0x7f6e94107990 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41358/0 (socket says 172.21.15.37:41358) 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 -- 172.21.15.37:0/3916364494 learned_addr learned my addr 172.21.15.37:0/3916364494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e99fea700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 0x7f6e941990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:21.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e9a7eb700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94107990 0x7f6e9419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 -- 172.21.15.37:0/3916364494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94107990 msgr2=0x7f6e9419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94107990 0x7f6e9419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 -- 172.21.15.37:0/3916364494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 msgr2=0x7f6e941990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 0x7f6e941990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.668+0000 7f6e997e9700 1 -- 172.21.15.37:0/3916364494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9419e0e0 con 0x7f6e94102300 2024-04-25T13:25:21.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e997e9700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400d980 tx=0x7f6e8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84016070 con 0x7f6e94102300 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9419e3d0 con 0x7f6e94102300 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e84004830 con 0x7f6e94102300 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e840054c0 con 0x7f6e94102300 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.669+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e941a5c50 con 0x7f6e94102300 2024-04-25T13:25:21.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.670+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e94069730 con 0x7f6e94102300 2024-04-25T13:25:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.671+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6e84004220 con 0x7f6e94102300 2024-04-25T13:25:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.672+0000 7f6e8affd700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 0x7f6e800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.672+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f6e840d0410 con 0x7f6e94102300 2024-04-25T13:25:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.672+0000 7f6e99fea700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 0x7f6e800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:21.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.673+0000 7f6e99fea700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 0x7f6e800a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000b5c0 tx=0x7f6e90005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:21.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.677+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e8401f020 con 0x7f6e94102300 2024-04-25T13:25:21.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:21 smithi037 ceph-mon[29204]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:21 smithi069 ceph-mon[32534]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:21.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:21 smithi044 ceph-mon[32590]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.890+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6e941a5e20 con 0x7f6e94102300 2024-04-25T13:25:21.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.891+0000 7f6e8affd700 1 -- 172.21.15.37:0/3916364494 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6e8409cc10 con 0x7f6e94102300 2024-04-25T13:25:21.891 INFO:teuthology.orchestra.run.smithi037.stdout:231928234032 2024-04-25T13:25:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 msgr2=0x7f6e800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:21.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 0x7f6e800a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000b5c0 tx=0x7f6e90005960 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 msgr2=0x7f6e94199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400d980 tx=0x7f6e8400dc90 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 shutdown_connections 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6e800a3d50 0x7f6e800a6210 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e94101940 0x7f6e941990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e94107990 0x7f6e9419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 --2- 172.21.15.37:0/3916364494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e94102300 0x7f6e94199610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 >> 172.21.15.37:0/3916364494 conn(0x7f6e940788f0 msgr2=0x7f6e9406a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 shutdown_connections 2024-04-25T13:25:21.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:21.893+0000 7f6e9c24e700 1 -- 172.21.15.37:0/3916364494 wait complete. 2024-04-25T13:25:22.837 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:22 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3916364494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:25:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:22 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3916364494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:25:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:22 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3916364494' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T13:25:23.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 -- 172.21.15.37:0/1160400131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8101930 msgr2=0x7ff2d8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 --2- 172.21.15.37:0/1160400131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8101930 0x7ff2d8101db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009990 tx=0x7ff2c0009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:23 smithi037 ceph-mon[29204]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 -- 172.21.15.37:0/1160400131 shutdown_connections 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 --2- 172.21.15.37:0/1160400131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d8107980 0x7ff2d8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 --2- 172.21.15.37:0/1160400131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d81022f0 0x7ff2d810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 --2- 172.21.15.37:0/1160400131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8101930 0x7ff2d8101db0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 -- 172.21.15.37:0/1160400131 >> 172.21.15.37:0/1160400131 conn(0x7ff2d80788b0 msgr2=0x7ff2d8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 -- 172.21.15.37:0/1160400131 shutdown_connections 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.755+0000 7ff2dde34700 1 -- 172.21.15.37:0/1160400131 wait complete. 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.756+0000 7ff2dde34700 1 Processor -- start 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.756+0000 7ff2dde34700 1 -- start start 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.756+0000 7ff2dde34700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 0x7ff2d8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2dde34700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2dde34700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 0x7ff2d819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2dde34700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8111ad0 con 0x7ff2d8107980 2024-04-25T13:25:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2dde34700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8111950 con 0x7ff2d81022f0 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2dde34700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8111c50 con 0x7ff2d8101930 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 0x7ff2d819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:35790/0 (socket says 172.21.15.37:35790) 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 -- 172.21.15.37:0/699611156 learned_addr learned my addr 172.21.15.37:0/699611156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d77fe700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 0x7ff2d8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 -- 172.21.15.37:0/699611156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 msgr2=0x7ff2d8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 0x7ff2d8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 -- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 msgr2=0x7ff2d819d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 0x7ff2d819d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 -- 172.21.15.37:0/699611156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d819e0d0 con 0x7ff2d81022f0 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d77fe700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 0x7ff2d8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d7fff700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 0x7ff2d819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:23.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.757+0000 7ff2d6ffd700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0000c00 tx=0x7ff2c00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.758+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0027070 con 0x7ff2d81022f0 2024-04-25T13:25:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.758+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d819e360 con 0x7ff2d81022f0 2024-04-25T13:25:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.758+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0003f20 con 0x7ff2d81022f0 2024-04-25T13:25:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.758+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0022ca0 con 0x7ff2d81022f0 2024-04-25T13:25:23.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.758+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a5c40 con 0x7ff2d81022f0 2024-04-25T13:25:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.759+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d8069730 con 0x7ff2d81022f0 2024-04-25T13:25:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.761+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff2c002a070 con 0x7ff2d81022f0 2024-04-25T13:25:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.761+0000 7ff2d4ff9700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 0x7ff2c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.762+0000 7ff2d77fe700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 0x7ff2c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.762+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7ff2c00d1490 con 0x7ff2d81022f0 2024-04-25T13:25:23.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.762+0000 7ff2d77fe700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 0x7ff2c40a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005950 tx=0x7ff2c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:23.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.765+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2c0013070 con 0x7ff2d81022f0 2024-04-25T13:25:23.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:23 smithi069 ceph-mon[32534]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:23.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:23 smithi044 ceph-mon[32590]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s; 4.9 KiB/s, 0 keys/s, 0 objects/s recovering 2024-04-25T13:25:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.985+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff2d8103410 con 0x7ff2d81022f0 2024-04-25T13:25:23.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.986+0000 7ff2d4ff9700 1 -- 172.21.15.37:0/699611156 <== mon.2 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff2c009dd20 con 0x7ff2d81022f0 2024-04-25T13:25:23.987 INFO:teuthology.orchestra.run.smithi037.stdout:171798691898 2024-04-25T13:25:23.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 msgr2=0x7ff2c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 0x7ff2c40a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005950 tx=0x7ff2c800b410 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 msgr2=0x7ff2d8199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0000c00 tx=0x7ff2c00047b0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 shutdown_connections 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ff2c40a3cd0 0x7ff2c40a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2d8107980 0x7ff2d819d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2d8101930 0x7ff2d8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 --2- 172.21.15.37:0/699611156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d81022f0 0x7ff2d8199650 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 >> 172.21.15.37:0/699611156 conn(0x7ff2d80788b0 msgr2=0x7ff2d8105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 shutdown_connections 2024-04-25T13:25:23.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7ff2dde34700 1 -- 172.21.15.37:0/699611156 wait complete. 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 -- 172.21.15.37:0/2040283060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774075590 msgr2=0x7fb77410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 --2- 172.21.15.37:0/2040283060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774075590 0x7fb77410ef00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb764009990 tx=0x7fb764009ca0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 -- 172.21.15.37:0/2040283060 shutdown_connections 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 --2- 172.21.15.37:0/2040283060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774074c70 0x7fb774075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 --2- 172.21.15.37:0/2040283060 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 0x7fb774111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 --2- 172.21.15.37:0/2040283060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774075590 0x7fb77410ef00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.988+0000 7fb77c020700 1 -- 172.21.15.37:0/2040283060 >> 172.21.15.37:0/2040283060 conn(0x7fb774079bf0 msgr2=0x7fb77407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7fb77c020700 1 -- 172.21.15.37:0/2040283060 shutdown_connections 2024-04-25T13:25:23.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7fb77c020700 1 -- 172.21.15.37:0/2040283060 wait complete. 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7fb77c020700 1 Processor -- start 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.989+0000 7fb77c020700 1 -- start start 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774075590 0x7fb77410be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 0x7fb77410c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb774114e40 con 0x7fb774074c70 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb774114cc0 con 0x7fb774075590 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77c020700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb774114fc0 con 0x7fb77410f440 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41378/0 (socket says 172.21.15.37:41378) 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 -- 172.21.15.37:0/734211356 learned_addr learned my addr 172.21.15.37:0/734211356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb77a5bd700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 0x7fb77410c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb7795bb700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774075590 0x7fb77410be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 -- 172.21.15.37:0/734211356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 msgr2=0x7fb77410c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 0x7fb77410c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 -- 172.21.15.37:0/734211356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774075590 msgr2=0x7fb77410be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774075590 0x7fb77410be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 -- 172.21.15.37:0/734211356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741abb80 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.990+0000 7fb779dbc700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb77000c9b0 tx=0x7fb77000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.991+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770007b00 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.991+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741abd50 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.991+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb77000ce50 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.991+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770018660 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.991+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741ac250 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.992+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774076390 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.993+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb77000f450 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.994+0000 7fb76affd700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 0x7fb7600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.994+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fb7700d0710 con 0x7fb774074c70 2024-04-25T13:25:23.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.994+0000 7fb7795bb700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 0x7fb7600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:23.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.995+0000 7fb7795bb700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 0x7fb7600a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b5c0 tx=0x7fb764005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:23.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:23.998+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb770014030 con 0x7fb774074c70 2024-04-25T13:25:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.212+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb77402df80 con 0x7fb774074c70 2024-04-25T13:25:24.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.212+0000 7fb76affd700 1 -- 172.21.15.37:0/734211356 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb77009d080 con 0x7fb774074c70 2024-04-25T13:25:24.213 INFO:teuthology.orchestra.run.smithi037.stdout:304942678054 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.214+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 msgr2=0x7fb7600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.214+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 0x7fb7600a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb76400b5c0 tx=0x7fb764005960 comp rx=0 tx=0).stop 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.214+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 msgr2=0x7fb77410eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.214+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb77000c9b0 tx=0x7fb77000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 shutdown_connections 2024-04-25T13:25:24.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb7600a3d50 0x7fb7600a6210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb77410f440 0x7fb77410c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb774074c70 0x7fb77410eda0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 --2- 172.21.15.37:0/734211356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb774075590 0x7fb77410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 >> 172.21.15.37:0/734211356 conn(0x7fb774079bf0 msgr2=0x7fb7740ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 shutdown_connections 2024-04-25T13:25:24.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:24.215+0000 7fb77c020700 1 -- 172.21.15.37:0/734211356 wait complete. 2024-04-25T13:25:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:24 smithi037 ceph-mon[29204]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:24 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/699611156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:25:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:24 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/734211356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:25:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:24 smithi069 ceph-mon[32534]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:24 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/699611156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:25:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:24 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/734211356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:25:24.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:24 smithi044 ceph-mon[32590]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:24.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:24 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/699611156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T13:25:24.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:24 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/734211356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T13:25:24.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678048 got 304942678054 for osd.9 2024-04-25T13:25:24.989 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:25.440 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691893 got 171798691898 for osd.5 2024-04-25T13:25:25.440 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:26.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234027 got 231928234032 for osd.7 2024-04-25T13:25:26.033 DEBUG:teuthology.parallel:result is None 2024-04-25T13:25:26.033 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T13:25:26.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:25:26.329 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:26 smithi037 ceph-mon[29204]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:26 smithi069 ceph-mon[32534]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:26 smithi044 ceph-mon[32590]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- 172.21.15.37:0/4165753500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806b980 msgr2=0x7f92c810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/4165753500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806b980 0x7f92c810f8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f92c000b0a0 tx=0x7f92c000b3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- 172.21.15.37:0/4165753500 shutdown_connections 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/4165753500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c8108a20 0x7f92c8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/4165753500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806b980 0x7f92c810f8f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/4165753500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c806afe0 0x7f92c806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- 172.21.15.37:0/4165753500 >> 172.21.15.37:0/4165753500 conn(0x7f92c80788b0 msgr2=0x7f92c8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- 172.21.15.37:0/4165753500 shutdown_connections 2024-04-25T13:25:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- 172.21.15.37:0/4165753500 wait complete. 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 Processor -- start 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.956+0000 7f92cfbf1700 1 -- start start 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c806b980 0x7f92c8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c8108a20 0x7f92c819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8112ba0 con 0x7f92c806afe0 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8112a20 con 0x7f92c806b980 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cfbf1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8112d20 con 0x7f92c8108a20 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41398/0 (socket says 172.21.15.37:41398) 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 -- 172.21.15.37:0/1042224248 learned_addr learned my addr 172.21.15.37:0/1042224248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd18c700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c806b980 0x7f92c8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92ce18e700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c8108a20 0x7f92c819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 -- 172.21.15.37:0/1042224248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c8108a20 msgr2=0x7f92c819d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c8108a20 0x7f92c819d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 -- 172.21.15.37:0/1042224248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c806b980 msgr2=0x7f92c8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c806b980 0x7f92c8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 -- 172.21.15.37:0/1042224248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c819e020 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.957+0000 7f92cd98d700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c9b0 tx=0x7f92c400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.958+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4007b00 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.958+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c400ce50 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.958+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c819e310 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.958+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4018660 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.958+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a5b90 con 0x7f92c806afe0 2024-04-25T13:25:27.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.959+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c8069730 con 0x7f92c806afe0 2024-04-25T13:25:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.960+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f92c400f450 con 0x7f92c806afe0 2024-04-25T13:25:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.960+0000 7f92baffd700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 0x7f92b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.961+0000 7f92cd18c700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 0x7f92b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.961+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f92c40d06f0 con 0x7f92c806afe0 2024-04-25T13:25:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.961+0000 7f92cd18c700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 0x7f92b40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92c8078e60 tx=0x7f92bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:27.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:27.964+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92c4014030 con 0x7f92c806afe0 2024-04-25T13:25:28.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.171+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92c8062e80 con 0x7f92b40a3cd0 2024-04-25T13:25:28.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.179+0000 7f92baffd700 1 -- 172.21.15.37:0/1042224248 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258650 (secure 0 0 0) 0x7f92c8062e80 con 0x7f92b40a3cd0 2024-04-25T13:25:28.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 msgr2=0x7f92b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 0x7f92b40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92c8078e60 tx=0x7f92bc00b410 comp rx=0 tx=0).stop 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 msgr2=0x7f92c8199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f92c400c9b0 tx=0x7f92c400ccc0 comp rx=0 tx=0).stop 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 shutdown_connections 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f92b40a3cd0 0x7f92b40a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:28.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92c8108a20 0x7f92c819d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c806afe0 0x7f92c8199010 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 --2- 172.21.15.37:0/1042224248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92c806b980 0x7f92c8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 >> 172.21.15.37:0/1042224248 conn(0x7f92c80788b0 msgr2=0x7f92c80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 shutdown_connections 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:28.183+0000 7f92cfbf1700 1 -- 172.21.15.37:0/1042224248 wait complete. 2024-04-25T13:25:28.187 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T13:25:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:28.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:28.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:28.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:28 smithi037 ceph-mon[29204]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:28.892 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-25T13:25:27.748098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":273,"num_read_kb":250,"num_write":259,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1241268,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412744,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529080,"statfs":{"total":1151940427776,"available":1151517777920,"internally_reserved":0,"allocated":13922304,"data_stored":7639500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532862+0000","last_change":"2024-04-25T13:24:11.873737+0000","last_active":"2024-04-25T13:25:07.532862+0000","last_peered":"2024-04-25T13:25:07.532862+0000","last_clean":"2024-04-25T13:25:07.532862+0000","last_became_active":"2024-04-25T13:24:11.872598+0000","last_became_peered":"2024-04-25T13:24:11.872598+0000","last_unstale":"2024-04-25T13:25:07.532862+0000","last_undegraded":"2024-04-25T13:25:07.532862+0000","last_fullsized":"2024-04-25T13:25:07.532862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:46.739869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530242+0000","last_change":"2024-04-25T13:24:11.886834+0000","last_active":"2024-04-25T13:25:07.530242+0000","last_peered":"2024-04-25T13:25:07.530242+0000","last_clean":"2024-04-25T13:25:07.530242+0000","last_became_active":"2024-04-25T13:24:11.886460+0000","last_became_peered":"2024-04-25T13:24:11.886460+0000","last_unstale":"2024-04-25T13:25:07.530242+0000","last_undegraded":"2024-04-25T13:25:07.530242+0000","last_fullsized":"2024-04-25T13:25:07.530242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.103106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530814+0000","last_change":"2024-04-25T13:24:11.877582+0000","last_active":"2024-04-25T13:25:07.530814+0000","last_peered":"2024-04-25T13:25:07.530814+0000","last_clean":"2024-04-25T13:25:07.530814+0000","last_became_active":"2024-04-25T13:24:11.877245+0000","last_became_peered":"2024-04-25T13:24:11.877245+0000","last_unstale":"2024-04-25T13:25:07.530814+0000","last_undegraded":"2024-04-25T13:25:07.530814+0000","last_fullsized":"2024-04-25T13:25:07.530814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:05.065255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529356+0000","last_change":"2024-04-25T13:24:11.873522+0000","last_active":"2024-04-25T13:25:07.529356+0000","last_peered":"2024-04-25T13:25:07.529356+0000","last_clean":"2024-04-25T13:25:07.529356+0000","last_became_active":"2024-04-25T13:24:11.872505+0000","last_became_peered":"2024-04-25T13:24:11.872505+0000","last_unstale":"2024-04-25T13:25:07.529356+0000","last_undegraded":"2024-04-25T13:25:07.529356+0000","last_fullsized":"2024-04-25T13:25:07.529356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:15.968935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535050+0000","last_change":"2024-04-25T13:24:11.873353+0000","last_active":"2024-04-25T13:25:07.535050+0000","last_peered":"2024-04-25T13:25:07.535050+0000","last_clean":"2024-04-25T13:25:07.535050+0000","last_became_active":"2024-04-25T13:24:11.872997+0000","last_became_peered":"2024-04-25T13:24:11.872997+0000","last_unstale":"2024-04-25T13:25:07.535050+0000","last_undegraded":"2024-04-25T13:25:07.535050+0000","last_fullsized":"2024-04-25T13:25:07.535050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:15.205105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531291+0000","last_change":"2024-04-25T13:24:11.875994+0000","last_active":"2024-04-25T13:25:07.531291+0000","last_peered":"2024-04-25T13:25:07.531291+0000","last_clean":"2024-04-25T13:25:07.531291+0000","last_became_active":"2024-04-25T13:24:11.875658+0000","last_became_peered":"2024-04-25T13:24:11.875658+0000","last_unstale":"2024-04-25T13:25:07.531291+0000","last_undegraded":"2024-04-25T13:25:07.531291+0000","last_fullsized":"2024-04-25T13:25:07.531291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:20.959715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534078+0000","last_change":"2024-04-25T13:24:11.879417+0000","last_active":"2024-04-25T13:25:07.534078+0000","last_peered":"2024-04-25T13:25:07.534078+0000","last_clean":"2024-04-25T13:25:07.534078+0000","last_became_active":"2024-04-25T13:24:11.879112+0000","last_became_peered":"2024-04-25T13:24:11.879112+0000","last_unstale":"2024-04-25T13:25:07.534078+0000","last_undegraded":"2024-04-25T13:25:07.534078+0000","last_fullsized":"2024-04-25T13:25:07.534078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:25.250253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531632+0000","last_change":"2024-04-25T13:24:11.883661+0000","last_active":"2024-04-25T13:25:07.531632+0000","last_peered":"2024-04-25T13:25:07.531632+0000","last_clean":"2024-04-25T13:25:07.531632+0000","last_became_active":"2024-04-25T13:24:11.883219+0000","last_became_peered":"2024-04-25T13:24:11.883219+0000","last_unstale":"2024-04-25T13:25:07.531632+0000","last_undegraded":"2024-04-25T13:25:07.531632+0000","last_fullsized":"2024-04-25T13:25:07.531632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:15.731727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530817+0000","last_change":"2024-04-25T13:24:11.880350+0000","last_active":"2024-04-25T13:25:07.530817+0000","last_peered":"2024-04-25T13:25:07.530817+0000","last_clean":"2024-04-25T13:25:07.530817+0000","last_became_active":"2024-04-25T13:24:11.880028+0000","last_became_peered":"2024-04-25T13:24:11.880028+0000","last_unstale":"2024-04-25T13:25:07.530817+0000","last_undegraded":"2024-04-25T13:25:07.530817+0000","last_fullsized":"2024-04-25T13:25:07.530817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:54.832380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531814+0000","last_change":"2024-04-25T13:24:11.879180+0000","last_active":"2024-04-25T13:25:07.531814+0000","last_peered":"2024-04-25T13:25:07.531814+0000","last_clean":"2024-04-25T13:25:07.531814+0000","last_became_active":"2024-04-25T13:24:11.877612+0000","last_became_peered":"2024-04-25T13:24:11.877612+0000","last_unstale":"2024-04-25T13:25:07.531814+0000","last_undegraded":"2024-04-25T13:25:07.531814+0000","last_fullsized":"2024-04-25T13:25:07.531814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.269957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535005+0000","last_change":"2024-04-25T13:24:11.880768+0000","last_active":"2024-04-25T13:25:07.535005+0000","last_peered":"2024-04-25T13:25:07.535005+0000","last_clean":"2024-04-25T13:25:07.535005+0000","last_became_active":"2024-04-25T13:24:11.879900+0000","last_became_peered":"2024-04-25T13:24:11.879900+0000","last_unstale":"2024-04-25T13:25:07.535005+0000","last_undegraded":"2024-04-25T13:25:07.535005+0000","last_fullsized":"2024-04-25T13:25:07.535005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:11.941761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529327+0000","last_change":"2024-04-25T13:24:11.873868+0000","last_active":"2024-04-25T13:25:07.529327+0000","last_peered":"2024-04-25T13:25:07.529327+0000","last_clean":"2024-04-25T13:25:07.529327+0000","last_became_active":"2024-04-25T13:24:11.872941+0000","last_became_peered":"2024-04-25T13:24:11.872941+0000","last_unstale":"2024-04-25T13:25:07.529327+0000","last_undegraded":"2024-04-25T13:25:07.529327+0000","last_fullsized":"2024-04-25T13:25:07.529327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:15.580750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562046+0000","last_change":"2024-04-25T13:24:11.888054+0000","last_active":"2024-04-25T13:25:07.562046+0000","last_peered":"2024-04-25T13:25:07.562046+0000","last_clean":"2024-04-25T13:25:07.562046+0000","last_became_active":"2024-04-25T13:24:11.887217+0000","last_became_peered":"2024-04-25T13:24:11.887217+0000","last_unstale":"2024-04-25T13:25:07.562046+0000","last_undegraded":"2024-04-25T13:25:07.562046+0000","last_fullsized":"2024-04-25T13:25:07.562046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:08.058645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530575+0000","last_change":"2024-04-25T13:24:11.878049+0000","last_active":"2024-04-25T13:25:07.530575+0000","last_peered":"2024-04-25T13:25:07.530575+0000","last_clean":"2024-04-25T13:25:07.530575+0000","last_became_active":"2024-04-25T13:24:11.876595+0000","last_became_peered":"2024-04-25T13:24:11.876595+0000","last_unstale":"2024-04-25T13:25:07.530575+0000","last_undegraded":"2024-04-25T13:25:07.530575+0000","last_fullsized":"2024-04-25T13:25:07.530575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:36.126016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530458+0000","last_change":"2024-04-25T13:24:11.890173+0000","last_active":"2024-04-25T13:25:07.530458+0000","last_peered":"2024-04-25T13:25:07.530458+0000","last_clean":"2024-04-25T13:25:07.530458+0000","last_became_active":"2024-04-25T13:24:11.889865+0000","last_became_peered":"2024-04-25T13:24:11.889865+0000","last_unstale":"2024-04-25T13:25:07.530458+0000","last_undegraded":"2024-04-25T13:25:07.530458+0000","last_fullsized":"2024-04-25T13:25:07.530458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:36.809979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530434+0000","last_change":"2024-04-25T13:24:11.888024+0000","last_active":"2024-04-25T13:25:07.530434+0000","last_peered":"2024-04-25T13:25:07.530434+0000","last_clean":"2024-04-25T13:25:07.530434+0000","last_became_active":"2024-04-25T13:24:11.887497+0000","last_became_peered":"2024-04-25T13:24:11.887497+0000","last_unstale":"2024-04-25T13:25:07.530434+0000","last_undegraded":"2024-04-25T13:25:07.530434+0000","last_fullsized":"2024-04-25T13:25:07.530434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.887282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531971+0000","last_change":"2024-04-25T13:24:11.872532+0000","last_active":"2024-04-25T13:25:07.531971+0000","last_peered":"2024-04-25T13:25:07.531971+0000","last_clean":"2024-04-25T13:25:07.531971+0000","last_became_active":"2024-04-25T13:24:11.871047+0000","last_became_peered":"2024-04-25T13:24:11.871047+0000","last_unstale":"2024-04-25T13:25:07.531971+0000","last_undegraded":"2024-04-25T13:25:07.531971+0000","last_fullsized":"2024-04-25T13:25:07.531971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:21.818593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530231+0000","last_change":"2024-04-25T13:24:11.869003+0000","last_active":"2024-04-25T13:25:07.530231+0000","last_peered":"2024-04-25T13:25:07.530231+0000","last_clean":"2024-04-25T13:25:07.530231+0000","last_became_active":"2024-04-25T13:24:11.868639+0000","last_became_peered":"2024-04-25T13:24:11.868639+0000","last_unstale":"2024-04-25T13:25:07.530231+0000","last_undegraded":"2024-04-25T13:25:07.530231+0000","last_fullsized":"2024-04-25T13:25:07.530231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:03.366661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528630+0000","last_change":"2024-04-25T13:24:11.878947+0000","last_active":"2024-04-25T13:25:07.528630+0000","last_peered":"2024-04-25T13:25:07.528630+0000","last_clean":"2024-04-25T13:25:07.528630+0000","last_became_active":"2024-04-25T13:24:11.878344+0000","last_became_peered":"2024-04-25T13:24:11.878344+0000","last_unstale":"2024-04-25T13:25:07.528630+0000","last_undegraded":"2024-04-25T13:25:07.528630+0000","last_fullsized":"2024-04-25T13:25:07.528630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:54.653873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530784+0000","last_change":"2024-04-25T13:24:10.856180+0000","last_active":"2024-04-25T13:25:07.530784+0000","last_peered":"2024-04-25T13:25:07.530784+0000","last_clean":"2024-04-25T13:25:07.530784+0000","last_became_active":"2024-04-25T13:24:10.855746+0000","last_became_peered":"2024-04-25T13:24:10.855746+0000","last_unstale":"2024-04-25T13:25:07.530784+0000","last_undegraded":"2024-04-25T13:25:07.530784+0000","last_fullsized":"2024-04-25T13:25:07.530784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:58.312713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532741+0000","last_change":"2024-04-25T13:24:11.889286+0000","last_active":"2024-04-25T13:25:07.532741+0000","last_peered":"2024-04-25T13:25:07.532741+0000","last_clean":"2024-04-25T13:25:07.532741+0000","last_became_active":"2024-04-25T13:24:11.888950+0000","last_became_peered":"2024-04-25T13:24:11.888950+0000","last_unstale":"2024-04-25T13:25:07.532741+0000","last_undegraded":"2024-04-25T13:25:07.532741+0000","last_fullsized":"2024-04-25T13:25:07.532741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:31.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.533022+0000","last_change":"2024-04-25T13:24:10.859008+0000","last_active":"2024-04-25T13:25:07.533022+0000","last_peered":"2024-04-25T13:25:07.533022+0000","last_clean":"2024-04-25T13:25:07.533022+0000","last_became_active":"2024-04-25T13:24:10.858350+0000","last_became_peered":"2024-04-25T13:24:10.858350+0000","last_unstale":"2024-04-25T13:25:07.533022+0000","last_undegraded":"2024-04-25T13:25:07.533022+0000","last_fullsized":"2024-04-25T13:25:07.533022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.048592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531248+0000","last_change":"2024-04-25T13:24:11.879185+0000","last_active":"2024-04-25T13:25:07.531248+0000","last_peered":"2024-04-25T13:25:07.531248+0000","last_clean":"2024-04-25T13:25:07.531248+0000","last_became_active":"2024-04-25T13:24:11.878808+0000","last_became_peered":"2024-04-25T13:24:11.878808+0000","last_unstale":"2024-04-25T13:25:07.531248+0000","last_undegraded":"2024-04-25T13:25:07.531248+0000","last_fullsized":"2024-04-25T13:25:07.531248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.408529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535898+0000","last_change":"2024-04-25T13:24:10.861742+0000","last_active":"2024-04-25T13:25:07.535898+0000","last_peered":"2024-04-25T13:25:07.535898+0000","last_clean":"2024-04-25T13:25:07.535898+0000","last_became_active":"2024-04-25T13:24:10.861302+0000","last_became_peered":"2024-04-25T13:24:10.861302+0000","last_unstale":"2024-04-25T13:25:07.535898+0000","last_undegraded":"2024-04-25T13:25:07.535898+0000","last_fullsized":"2024-04-25T13:25:07.535898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.470804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530803+0000","last_change":"2024-04-25T13:24:11.873872+0000","last_active":"2024-04-25T13:25:07.530803+0000","last_peered":"2024-04-25T13:25:07.530803+0000","last_clean":"2024-04-25T13:25:07.530803+0000","last_became_active":"2024-04-25T13:24:11.873341+0000","last_became_peered":"2024-04-25T13:24:11.873341+0000","last_unstale":"2024-04-25T13:25:07.530803+0000","last_undegraded":"2024-04-25T13:25:07.530803+0000","last_fullsized":"2024-04-25T13:25:07.530803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:46.271742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535165+0000","last_change":"2024-04-25T13:24:10.856554+0000","last_active":"2024-04-25T13:25:07.535165+0000","last_peered":"2024-04-25T13:25:07.535165+0000","last_clean":"2024-04-25T13:25:07.535165+0000","last_became_active":"2024-04-25T13:24:10.856254+0000","last_became_peered":"2024-04-25T13:24:10.856254+0000","last_unstale":"2024-04-25T13:25:07.535165+0000","last_undegraded":"2024-04-25T13:25:07.535165+0000","last_fullsized":"2024-04-25T13:25:07.535165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:15.843985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530992+0000","last_change":"2024-04-25T13:24:11.891970+0000","last_active":"2024-04-25T13:25:07.530992+0000","last_peered":"2024-04-25T13:25:07.530992+0000","last_clean":"2024-04-25T13:25:07.530992+0000","last_became_active":"2024-04-25T13:24:11.891537+0000","last_became_peered":"2024-04-25T13:24:11.891537+0000","last_unstale":"2024-04-25T13:25:07.530992+0000","last_undegraded":"2024-04-25T13:25:07.530992+0000","last_fullsized":"2024-04-25T13:25:07.530992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.509206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530789+0000","last_change":"2024-04-25T13:24:10.857130+0000","last_active":"2024-04-25T13:25:07.530789+0000","last_peered":"2024-04-25T13:25:07.530789+0000","last_clean":"2024-04-25T13:25:07.530789+0000","last_became_active":"2024-04-25T13:24:10.856747+0000","last_became_peered":"2024-04-25T13:24:10.856747+0000","last_unstale":"2024-04-25T13:25:07.530789+0000","last_undegraded":"2024-04-25T13:25:07.530789+0000","last_fullsized":"2024-04-25T13:25:07.530789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:11.366223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534986+0000","last_change":"2024-04-25T13:24:11.873404+0000","last_active":"2024-04-25T13:25:07.534986+0000","last_peered":"2024-04-25T13:25:07.534986+0000","last_clean":"2024-04-25T13:25:07.534986+0000","last_became_active":"2024-04-25T13:24:11.873027+0000","last_became_peered":"2024-04-25T13:24:11.873027+0000","last_unstale":"2024-04-25T13:25:07.534986+0000","last_undegraded":"2024-04-25T13:25:07.534986+0000","last_fullsized":"2024-04-25T13:25:07.534986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:32.789300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531172+0000","last_change":"2024-04-25T13:24:10.853602+0000","last_active":"2024-04-25T13:25:07.531172+0000","last_peered":"2024-04-25T13:25:07.531172+0000","last_clean":"2024-04-25T13:25:07.531172+0000","last_became_active":"2024-04-25T13:24:10.853237+0000","last_became_peered":"2024-04-25T13:24:10.853237+0000","last_unstale":"2024-04-25T13:25:07.531172+0000","last_undegraded":"2024-04-25T13:25:07.531172+0000","last_fullsized":"2024-04-25T13:25:07.531172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.575835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562946+0000","last_change":"2024-04-25T13:25:07.562946+0000","last_active":"2024-04-25T13:25:07.562946+0000","last_peered":"2024-04-25T13:25:07.562946+0000","last_clean":"2024-04-25T13:25:07.562946+0000","last_became_active":"2024-04-25T13:25:07.562651+0000","last_became_peered":"2024-04-25T13:25:07.562651+0000","last_unstale":"2024-04-25T13:25:07.562946+0000","last_undegraded":"2024-04-25T13:25:07.562946+0000","last_fullsized":"2024-04-25T13:25:07.562946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:34.377961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561811+0000","last_change":"2024-04-25T13:24:10.851042+0000","last_active":"2024-04-25T13:25:07.561811+0000","last_peered":"2024-04-25T13:25:07.561811+0000","last_clean":"2024-04-25T13:25:07.561811+0000","last_became_active":"2024-04-25T13:24:10.850619+0000","last_became_peered":"2024-04-25T13:24:10.850619+0000","last_unstale":"2024-04-25T13:25:07.561811+0000","last_undegraded":"2024-04-25T13:25:07.561811+0000","last_fullsized":"2024-04-25T13:25:07.561811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:23.577971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531694+0000","last_change":"2024-04-25T13:24:11.880170+0000","last_active":"2024-04-25T13:25:07.531694+0000","last_peered":"2024-04-25T13:25:07.531694+0000","last_clean":"2024-04-25T13:25:07.531694+0000","last_became_active":"2024-04-25T13:24:11.879802+0000","last_became_peered":"2024-04-25T13:24:11.879802+0000","last_unstale":"2024-04-25T13:25:07.531694+0000","last_undegraded":"2024-04-25T13:25:07.531694+0000","last_fullsized":"2024-04-25T13:25:07.531694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:25.231071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530344+0000","last_change":"2024-04-25T13:24:10.852667+0000","last_active":"2024-04-25T13:25:07.530344+0000","last_peered":"2024-04-25T13:25:07.530344+0000","last_clean":"2024-04-25T13:25:07.530344+0000","last_became_active":"2024-04-25T13:24:10.852222+0000","last_became_peered":"2024-04-25T13:24:10.852222+0000","last_unstale":"2024-04-25T13:25:07.530344+0000","last_undegraded":"2024-04-25T13:25:07.530344+0000","last_fullsized":"2024-04-25T13:25:07.530344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:06.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530027+0000","last_change":"2024-04-25T13:24:11.867984+0000","last_active":"2024-04-25T13:25:07.530027+0000","last_peered":"2024-04-25T13:25:07.530027+0000","last_clean":"2024-04-25T13:25:07.530027+0000","last_became_active":"2024-04-25T13:24:11.867517+0000","last_became_peered":"2024-04-25T13:24:11.867517+0000","last_unstale":"2024-04-25T13:25:07.530027+0000","last_undegraded":"2024-04-25T13:25:07.530027+0000","last_fullsized":"2024-04-25T13:25:07.530027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.540573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531860+0000","last_change":"2024-04-25T13:24:10.855925+0000","last_active":"2024-04-25T13:25:07.531860+0000","last_peered":"2024-04-25T13:25:07.531860+0000","last_clean":"2024-04-25T13:25:07.531860+0000","last_became_active":"2024-04-25T13:24:10.855603+0000","last_became_peered":"2024-04-25T13:24:10.855603+0000","last_unstale":"2024-04-25T13:25:07.531860+0000","last_undegraded":"2024-04-25T13:25:07.531860+0000","last_fullsized":"2024-04-25T13:25:07.531860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:26.796979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530912+0000","last_change":"2024-04-25T13:24:11.873683+0000","last_active":"2024-04-25T13:25:07.530912+0000","last_peered":"2024-04-25T13:25:07.530912+0000","last_clean":"2024-04-25T13:25:07.530912+0000","last_became_active":"2024-04-25T13:24:11.873307+0000","last_became_peered":"2024-04-25T13:24:11.873307+0000","last_unstale":"2024-04-25T13:25:07.530912+0000","last_undegraded":"2024-04-25T13:25:07.530912+0000","last_fullsized":"2024-04-25T13:25:07.530912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:44.305349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535141+0000","last_change":"2024-04-25T13:24:10.857124+0000","last_active":"2024-04-25T13:25:07.535141+0000","last_peered":"2024-04-25T13:25:07.535141+0000","last_clean":"2024-04-25T13:25:07.535141+0000","last_became_active":"2024-04-25T13:24:10.856396+0000","last_became_peered":"2024-04-25T13:24:10.856396+0000","last_unstale":"2024-04-25T13:25:07.535141+0000","last_undegraded":"2024-04-25T13:25:07.535141+0000","last_fullsized":"2024-04-25T13:25:07.535141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:58.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531165+0000","last_change":"2024-04-25T13:24:11.882305+0000","last_active":"2024-04-25T13:25:07.531165+0000","last_peered":"2024-04-25T13:25:07.531165+0000","last_clean":"2024-04-25T13:25:07.531165+0000","last_became_active":"2024-04-25T13:24:11.881976+0000","last_became_peered":"2024-04-25T13:24:11.881976+0000","last_unstale":"2024-04-25T13:25:07.531165+0000","last_undegraded":"2024-04-25T13:25:07.531165+0000","last_fullsized":"2024-04-25T13:25:07.531165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:46.254871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531049+0000","last_change":"2024-04-25T13:24:10.864302+0000","last_active":"2024-04-25T13:25:07.531049+0000","last_peered":"2024-04-25T13:25:07.531049+0000","last_clean":"2024-04-25T13:25:07.531049+0000","last_became_active":"2024-04-25T13:24:10.863955+0000","last_became_peered":"2024-04-25T13:24:10.863955+0000","last_unstale":"2024-04-25T13:25:07.531049+0000","last_undegraded":"2024-04-25T13:25:07.531049+0000","last_fullsized":"2024-04-25T13:25:07.531049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:35.394778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531042+0000","last_change":"2024-04-25T13:24:11.876293+0000","last_active":"2024-04-25T13:25:07.531042+0000","last_peered":"2024-04-25T13:25:07.531042+0000","last_clean":"2024-04-25T13:25:07.531042+0000","last_became_active":"2024-04-25T13:24:11.875927+0000","last_became_peered":"2024-04-25T13:24:11.875927+0000","last_unstale":"2024-04-25T13:25:07.531042+0000","last_undegraded":"2024-04-25T13:25:07.531042+0000","last_fullsized":"2024-04-25T13:25:07.531042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:47.884328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":209,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:24.566420+0000","last_change":"2024-04-25T13:25:07.543419+0000","last_active":"2024-04-25T13:25:24.566420+0000","last_peered":"2024-04-25T13:25:24.566420+0000","last_clean":"2024-04-25T13:25:24.566420+0000","last_became_active":"2024-04-25T13:25:07.535875+0000","last_became_peered":"2024-04-25T13:25:07.535875+0000","last_unstale":"2024-04-25T13:25:24.566420+0000","last_undegraded":"2024-04-25T13:25:24.566420+0000","last_fullsized":"2024-04-25T13:25:24.566420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:18.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":131,"num_read_kb":125,"num_write":27,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":60532,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535657+0000","last_change":"2024-04-25T13:24:11.886845+0000","last_active":"2024-04-25T13:25:07.535657+0000","last_peered":"2024-04-25T13:25:07.535657+0000","last_clean":"2024-04-25T13:25:07.535657+0000","last_became_active":"2024-04-25T13:24:11.885457+0000","last_became_peered":"2024-04-25T13:24:11.885457+0000","last_unstale":"2024-04-25T13:25:07.535657+0000","last_undegraded":"2024-04-25T13:25:07.535657+0000","last_fullsized":"2024-04-25T13:25:07.535657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.134026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528563+0000","last_change":"2024-04-25T13:24:10.865292+0000","last_active":"2024-04-25T13:25:07.528563+0000","last_peered":"2024-04-25T13:25:07.528563+0000","last_clean":"2024-04-25T13:25:07.528563+0000","last_became_active":"2024-04-25T13:24:10.864940+0000","last_became_peered":"2024-04-25T13:24:10.864940+0000","last_unstale":"2024-04-25T13:25:07.528563+0000","last_undegraded":"2024-04-25T13:25:07.528563+0000","last_fullsized":"2024-04-25T13:25:07.528563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:54.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561814+0000","last_change":"2024-04-25T13:24:11.885158+0000","last_active":"2024-04-25T13:25:07.561814+0000","last_peered":"2024-04-25T13:25:07.561814+0000","last_clean":"2024-04-25T13:25:07.561814+0000","last_became_active":"2024-04-25T13:24:11.884367+0000","last_became_peered":"2024-04-25T13:24:11.884367+0000","last_unstale":"2024-04-25T13:25:07.561814+0000","last_undegraded":"2024-04-25T13:25:07.561814+0000","last_fullsized":"2024-04-25T13:25:07.561814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:48.244922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530650+0000","last_change":"2024-04-25T13:24:10.854269+0000","last_active":"2024-04-25T13:25:07.530650+0000","last_peered":"2024-04-25T13:25:07.530650+0000","last_clean":"2024-04-25T13:25:07.530650+0000","last_became_active":"2024-04-25T13:24:10.853968+0000","last_became_peered":"2024-04-25T13:24:10.853968+0000","last_unstale":"2024-04-25T13:25:07.530650+0000","last_undegraded":"2024-04-25T13:25:07.530650+0000","last_fullsized":"2024-04-25T13:25:07.530650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:42.771743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534934+0000","last_change":"2024-04-25T13:24:11.887468+0000","last_active":"2024-04-25T13:25:07.534934+0000","last_peered":"2024-04-25T13:25:07.534934+0000","last_clean":"2024-04-25T13:25:07.534934+0000","last_became_active":"2024-04-25T13:24:11.887097+0000","last_became_peered":"2024-04-25T13:24:11.887097+0000","last_unstale":"2024-04-25T13:25:07.534934+0000","last_undegraded":"2024-04-25T13:25:07.534934+0000","last_fullsized":"2024-04-25T13:25:07.534934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:23.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528632+0000","last_change":"2024-04-25T13:24:10.854484+0000","last_active":"2024-04-25T13:25:07.528632+0000","last_peered":"2024-04-25T13:25:07.528632+0000","last_clean":"2024-04-25T13:25:07.528632+0000","last_became_active":"2024-04-25T13:24:10.854152+0000","last_became_peered":"2024-04-25T13:24:10.854152+0000","last_unstale":"2024-04-25T13:25:07.528632+0000","last_undegraded":"2024-04-25T13:25:07.528632+0000","last_fullsized":"2024-04-25T13:25:07.528632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:25.940414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534026+0000","last_change":"2024-04-25T13:24:11.880661+0000","last_active":"2024-04-25T13:25:07.534026+0000","last_peered":"2024-04-25T13:25:07.534026+0000","last_clean":"2024-04-25T13:25:07.534026+0000","last_became_active":"2024-04-25T13:24:11.880099+0000","last_became_peered":"2024-04-25T13:24:11.880099+0000","last_unstale":"2024-04-25T13:25:07.534026+0000","last_undegraded":"2024-04-25T13:25:07.534026+0000","last_fullsized":"2024-04-25T13:25:07.534026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534443+0000","last_change":"2024-04-25T13:24:10.863356+0000","last_active":"2024-04-25T13:25:07.534443+0000","last_peered":"2024-04-25T13:25:07.534443+0000","last_clean":"2024-04-25T13:25:07.534443+0000","last_became_active":"2024-04-25T13:24:10.863024+0000","last_became_peered":"2024-04-25T13:24:10.863024+0000","last_unstale":"2024-04-25T13:25:07.534443+0000","last_undegraded":"2024-04-25T13:25:07.534443+0000","last_fullsized":"2024-04-25T13:25:07.534443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:11.520794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530444+0000","last_change":"2024-04-25T13:24:11.891787+0000","last_active":"2024-04-25T13:25:07.530444+0000","last_peered":"2024-04-25T13:25:07.530444+0000","last_clean":"2024-04-25T13:25:07.530444+0000","last_became_active":"2024-04-25T13:24:11.891466+0000","last_became_peered":"2024-04-25T13:24:11.891466+0000","last_unstale":"2024-04-25T13:25:07.530444+0000","last_undegraded":"2024-04-25T13:25:07.530444+0000","last_fullsized":"2024-04-25T13:25:07.530444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:11.200558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535661+0000","last_change":"2024-04-25T13:24:10.852320+0000","last_active":"2024-04-25T13:25:07.535661+0000","last_peered":"2024-04-25T13:25:07.535661+0000","last_clean":"2024-04-25T13:25:07.535661+0000","last_became_active":"2024-04-25T13:24:10.852007+0000","last_became_peered":"2024-04-25T13:24:10.852007+0000","last_unstale":"2024-04-25T13:25:07.535661+0000","last_undegraded":"2024-04-25T13:25:07.535661+0000","last_fullsized":"2024-04-25T13:25:07.535661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.582461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534523+0000","last_change":"2024-04-25T13:25:07.534523+0000","last_active":"2024-04-25T13:25:07.534523+0000","last_peered":"2024-04-25T13:25:07.534523+0000","last_clean":"2024-04-25T13:25:07.534523+0000","last_became_active":"2024-04-25T13:25:07.534117+0000","last_became_peered":"2024-04-25T13:25:07.534117+0000","last_unstale":"2024-04-25T13:25:07.534523+0000","last_undegraded":"2024-04-25T13:25:07.534523+0000","last_fullsized":"2024-04-25T13:25:07.534523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:12.904043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530081+0000","last_change":"2024-04-25T13:24:10.857715+0000","last_active":"2024-04-25T13:25:07.530081+0000","last_peered":"2024-04-25T13:25:07.530081+0000","last_clean":"2024-04-25T13:25:07.530081+0000","last_became_active":"2024-04-25T13:24:10.857398+0000","last_became_peered":"2024-04-25T13:24:10.857398+0000","last_unstale":"2024-04-25T13:25:07.530081+0000","last_undegraded":"2024-04-25T13:25:07.530081+0000","last_fullsized":"2024-04-25T13:25:07.530081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:58.189652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531704+0000","last_change":"2024-04-25T13:24:11.875065+0000","last_active":"2024-04-25T13:25:07.531704+0000","last_peered":"2024-04-25T13:25:07.531704+0000","last_clean":"2024-04-25T13:25:07.531704+0000","last_became_active":"2024-04-25T13:24:11.874433+0000","last_became_peered":"2024-04-25T13:24:11.874433+0000","last_unstale":"2024-04-25T13:25:07.531704+0000","last_undegraded":"2024-04-25T13:25:07.531704+0000","last_fullsized":"2024-04-25T13:25:07.531704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:07.430007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534227+0000","last_change":"2024-04-25T13:24:10.850250+0000","last_active":"2024-04-25T13:25:07.534227+0000","last_peered":"2024-04-25T13:25:07.534227+0000","last_clean":"2024-04-25T13:25:07.534227+0000","last_became_active":"2024-04-25T13:24:10.849773+0000","last_became_peered":"2024-04-25T13:24:10.849773+0000","last_unstale":"2024-04-25T13:25:07.534227+0000","last_undegraded":"2024-04-25T13:25:07.534227+0000","last_fullsized":"2024-04-25T13:25:07.534227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:59.609421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531693+0000","last_change":"2024-04-25T13:24:11.878625+0000","last_active":"2024-04-25T13:25:07.531693+0000","last_peered":"2024-04-25T13:25:07.531693+0000","last_clean":"2024-04-25T13:25:07.531693+0000","last_became_active":"2024-04-25T13:24:11.878311+0000","last_became_peered":"2024-04-25T13:24:11.878311+0000","last_unstale":"2024-04-25T13:25:07.531693+0000","last_undegraded":"2024-04-25T13:25:07.531693+0000","last_fullsized":"2024-04-25T13:25:07.531693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.363441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530532+0000","last_change":"2024-04-25T13:24:11.887874+0000","last_active":"2024-04-25T13:25:07.530532+0000","last_peered":"2024-04-25T13:25:07.530532+0000","last_clean":"2024-04-25T13:25:07.530532+0000","last_became_active":"2024-04-25T13:24:11.887558+0000","last_became_peered":"2024-04-25T13:24:11.887558+0000","last_unstale":"2024-04-25T13:25:07.530532+0000","last_undegraded":"2024-04-25T13:25:07.530532+0000","last_fullsized":"2024-04-25T13:25:07.530532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:49.861447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561681+0000","last_change":"2024-04-25T13:24:10.859643+0000","last_active":"2024-04-25T13:25:07.561681+0000","last_peered":"2024-04-25T13:25:07.561681+0000","last_clean":"2024-04-25T13:25:07.561681+0000","last_became_active":"2024-04-25T13:24:10.859312+0000","last_became_peered":"2024-04-25T13:24:10.859312+0000","last_unstale":"2024-04-25T13:25:07.561681+0000","last_undegraded":"2024-04-25T13:25:07.561681+0000","last_fullsized":"2024-04-25T13:25:07.561681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.513203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532592+0000","last_change":"2024-04-25T13:24:11.883821+0000","last_active":"2024-04-25T13:25:07.532592+0000","last_peered":"2024-04-25T13:25:07.532592+0000","last_clean":"2024-04-25T13:25:07.532592+0000","last_became_active":"2024-04-25T13:24:11.883063+0000","last_became_peered":"2024-04-25T13:24:11.883063+0000","last_unstale":"2024-04-25T13:25:07.532592+0000","last_undegraded":"2024-04-25T13:25:07.532592+0000","last_fullsized":"2024-04-25T13:25:07.532592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.845694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529094+0000","last_change":"2024-04-25T13:24:10.851525+0000","last_active":"2024-04-25T13:25:07.529094+0000","last_peered":"2024-04-25T13:25:07.529094+0000","last_clean":"2024-04-25T13:25:07.529094+0000","last_became_active":"2024-04-25T13:24:10.851171+0000","last_became_peered":"2024-04-25T13:24:10.851171+0000","last_unstale":"2024-04-25T13:25:07.529094+0000","last_undegraded":"2024-04-25T13:25:07.529094+0000","last_fullsized":"2024-04-25T13:25:07.529094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534069+0000","last_change":"2024-04-25T13:22:58.125848+0000","last_active":"2024-04-25T13:25:07.534069+0000","last_peered":"2024-04-25T13:25:07.534069+0000","last_clean":"2024-04-25T13:25:07.534069+0000","last_became_active":"2024-04-25T13:22:58.125374+0000","last_became_peered":"2024-04-25T13:22:58.125374+0000","last_unstale":"2024-04-25T13:25:07.534069+0000","last_undegraded":"2024-04-25T13:25:07.534069+0000","last_fullsized":"2024-04-25T13:25:07.534069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_clean_scrub_stamp":"2024-04-25T13:19:36.251667+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.304605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561679+0000","last_change":"2024-04-25T13:24:11.883648+0000","last_active":"2024-04-25T13:25:07.561679+0000","last_peered":"2024-04-25T13:25:07.561679+0000","last_clean":"2024-04-25T13:25:07.561679+0000","last_became_active":"2024-04-25T13:24:11.879538+0000","last_became_peered":"2024-04-25T13:24:11.879538+0000","last_unstale":"2024-04-25T13:25:07.561679+0000","last_undegraded":"2024-04-25T13:25:07.561679+0000","last_fullsized":"2024-04-25T13:25:07.561679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537012+0000","last_change":"2024-04-25T13:25:07.537012+0000","last_active":"2024-04-25T13:25:07.537012+0000","last_peered":"2024-04-25T13:25:07.537012+0000","last_clean":"2024-04-25T13:25:07.537012+0000","last_became_active":"2024-04-25T13:25:07.536537+0000","last_became_peered":"2024-04-25T13:25:07.536537+0000","last_unstale":"2024-04-25T13:25:07.537012+0000","last_undegraded":"2024-04-25T13:25:07.537012+0000","last_fullsized":"2024-04-25T13:25:07.537012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:33.420293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535778+0000","last_change":"2024-04-25T13:24:10.864221+0000","last_active":"2024-04-25T13:25:07.535778+0000","last_peered":"2024-04-25T13:25:07.535778+0000","last_clean":"2024-04-25T13:25:07.535778+0000","last_became_active":"2024-04-25T13:24:10.863892+0000","last_became_peered":"2024-04-25T13:24:10.863892+0000","last_unstale":"2024-04-25T13:25:07.535778+0000","last_undegraded":"2024-04-25T13:25:07.535778+0000","last_fullsized":"2024-04-25T13:25:07.535778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:51.229865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561995+0000","last_change":"2024-04-25T13:24:11.878257+0000","last_active":"2024-04-25T13:25:07.561995+0000","last_peered":"2024-04-25T13:25:07.561995+0000","last_clean":"2024-04-25T13:25:07.561995+0000","last_became_active":"2024-04-25T13:24:11.877926+0000","last_became_peered":"2024-04-25T13:24:11.877926+0000","last_unstale":"2024-04-25T13:25:07.561995+0000","last_undegraded":"2024-04-25T13:25:07.561995+0000","last_fullsized":"2024-04-25T13:25:07.561995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:32.873011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.539670+0000","last_change":"2024-04-25T13:25:07.539670+0000","last_active":"2024-04-25T13:25:07.539670+0000","last_peered":"2024-04-25T13:25:07.539670+0000","last_clean":"2024-04-25T13:25:07.539670+0000","last_became_active":"2024-04-25T13:25:07.535378+0000","last_became_peered":"2024-04-25T13:25:07.535378+0000","last_unstale":"2024-04-25T13:25:07.539670+0000","last_undegraded":"2024-04-25T13:25:07.539670+0000","last_fullsized":"2024-04-25T13:25:07.539670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:31.843073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.536213+0000","last_change":"2024-04-25T13:25:07.535542+0000","last_active":"2024-04-25T13:25:07.536213+0000","last_peered":"2024-04-25T13:25:07.536213+0000","last_clean":"2024-04-25T13:25:07.536213+0000","last_became_active":"2024-04-25T13:25:07.535142+0000","last_became_peered":"2024-04-25T13:25:07.535142+0000","last_unstale":"2024-04-25T13:25:07.536213+0000","last_undegraded":"2024-04-25T13:25:07.536213+0000","last_fullsized":"2024-04-25T13:25:07.536213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.618062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529311+0000","last_change":"2024-04-25T13:24:10.854441+0000","last_active":"2024-04-25T13:25:07.529311+0000","last_peered":"2024-04-25T13:25:07.529311+0000","last_clean":"2024-04-25T13:25:07.529311+0000","last_became_active":"2024-04-25T13:24:10.854086+0000","last_became_peered":"2024-04-25T13:24:10.854086+0000","last_unstale":"2024-04-25T13:25:07.529311+0000","last_undegraded":"2024-04-25T13:25:07.529311+0000","last_fullsized":"2024-04-25T13:25:07.529311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:17.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530861+0000","last_change":"2024-04-25T13:24:11.880494+0000","last_active":"2024-04-25T13:25:07.530861+0000","last_peered":"2024-04-25T13:25:07.530861+0000","last_clean":"2024-04-25T13:25:07.530861+0000","last_became_active":"2024-04-25T13:24:11.880074+0000","last_became_peered":"2024-04-25T13:24:11.880074+0000","last_unstale":"2024-04-25T13:25:07.530861+0000","last_undegraded":"2024-04-25T13:25:07.530861+0000","last_fullsized":"2024-04-25T13:25:07.530861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:50.993165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535837+0000","last_change":"2024-04-25T13:25:07.535837+0000","last_active":"2024-04-25T13:25:07.535837+0000","last_peered":"2024-04-25T13:25:07.535837+0000","last_clean":"2024-04-25T13:25:07.535837+0000","last_became_active":"2024-04-25T13:25:07.535415+0000","last_became_peered":"2024-04-25T13:25:07.535415+0000","last_unstale":"2024-04-25T13:25:07.535837+0000","last_undegraded":"2024-04-25T13:25:07.535837+0000","last_fullsized":"2024-04-25T13:25:07.535837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534583+0000","last_change":"2024-04-25T13:24:11.878770+0000","last_active":"2024-04-25T13:25:07.534583+0000","last_peered":"2024-04-25T13:25:07.534583+0000","last_clean":"2024-04-25T13:25:07.534583+0000","last_became_active":"2024-04-25T13:24:11.878424+0000","last_became_peered":"2024-04-25T13:24:11.878424+0000","last_unstale":"2024-04-25T13:25:07.534583+0000","last_undegraded":"2024-04-25T13:25:07.534583+0000","last_fullsized":"2024-04-25T13:25:07.534583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:25.016200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531910+0000","last_change":"2024-04-25T13:24:10.857246+0000","last_active":"2024-04-25T13:25:07.531910+0000","last_peered":"2024-04-25T13:25:07.531910+0000","last_clean":"2024-04-25T13:25:07.531910+0000","last_became_active":"2024-04-25T13:24:10.856922+0000","last_became_peered":"2024-04-25T13:24:10.856922+0000","last_unstale":"2024-04-25T13:25:07.531910+0000","last_undegraded":"2024-04-25T13:25:07.531910+0000","last_fullsized":"2024-04-25T13:25:07.531910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:08.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531105+0000","last_change":"2024-04-25T13:24:11.873225+0000","last_active":"2024-04-25T13:25:07.531105+0000","last_peered":"2024-04-25T13:25:07.531105+0000","last_clean":"2024-04-25T13:25:07.531105+0000","last_became_active":"2024-04-25T13:24:11.872855+0000","last_became_peered":"2024-04-25T13:24:11.872855+0000","last_unstale":"2024-04-25T13:25:07.531105+0000","last_undegraded":"2024-04-25T13:25:07.531105+0000","last_fullsized":"2024-04-25T13:25:07.531105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.523621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528331+0000","last_change":"2024-04-25T13:24:10.863165+0000","last_active":"2024-04-25T13:25:07.528331+0000","last_peered":"2024-04-25T13:25:07.528331+0000","last_clean":"2024-04-25T13:25:07.528331+0000","last_became_active":"2024-04-25T13:24:10.862596+0000","last_became_peered":"2024-04-25T13:24:10.862596+0000","last_unstale":"2024-04-25T13:25:07.528331+0000","last_undegraded":"2024-04-25T13:25:07.528331+0000","last_fullsized":"2024-04-25T13:25:07.528331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:51.414158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532355+0000","last_change":"2024-04-25T13:24:11.885852+0000","last_active":"2024-04-25T13:25:07.532355+0000","last_peered":"2024-04-25T13:25:07.532355+0000","last_clean":"2024-04-25T13:25:07.532355+0000","last_became_active":"2024-04-25T13:24:11.885508+0000","last_became_peered":"2024-04-25T13:24:11.885508+0000","last_unstale":"2024-04-25T13:25:07.532355+0000","last_undegraded":"2024-04-25T13:25:07.532355+0000","last_fullsized":"2024-04-25T13:25:07.532355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.575351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535089+0000","last_change":"2024-04-25T13:24:10.861701+0000","last_active":"2024-04-25T13:25:07.535089+0000","last_peered":"2024-04-25T13:25:07.535089+0000","last_clean":"2024-04-25T13:25:07.535089+0000","last_became_active":"2024-04-25T13:24:10.861356+0000","last_became_peered":"2024-04-25T13:24:10.861356+0000","last_unstale":"2024-04-25T13:25:07.535089+0000","last_undegraded":"2024-04-25T13:25:07.535089+0000","last_fullsized":"2024-04-25T13:25:07.535089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:24.300835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534372+0000","last_change":"2024-04-25T13:24:11.883977+0000","last_active":"2024-04-25T13:25:07.534372+0000","last_peered":"2024-04-25T13:25:07.534372+0000","last_clean":"2024-04-25T13:25:07.534372+0000","last_became_active":"2024-04-25T13:24:11.883401+0000","last_became_peered":"2024-04-25T13:24:11.883401+0000","last_unstale":"2024-04-25T13:25:07.534372+0000","last_undegraded":"2024-04-25T13:25:07.534372+0000","last_fullsized":"2024-04-25T13:25:07.534372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:04.291865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528716+0000","last_change":"2024-04-25T13:24:10.858303+0000","last_active":"2024-04-25T13:25:07.528716+0000","last_peered":"2024-04-25T13:25:07.528716+0000","last_clean":"2024-04-25T13:25:07.528716+0000","last_became_active":"2024-04-25T13:24:10.857956+0000","last_became_peered":"2024-04-25T13:24:10.857956+0000","last_unstale":"2024-04-25T13:25:07.528716+0000","last_undegraded":"2024-04-25T13:25:07.528716+0000","last_fullsized":"2024-04-25T13:25:07.528716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.753759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535087+0000","last_change":"2024-04-25T13:24:11.882367+0000","last_active":"2024-04-25T13:25:07.535087+0000","last_peered":"2024-04-25T13:25:07.535087+0000","last_clean":"2024-04-25T13:25:07.535087+0000","last_became_active":"2024-04-25T13:24:11.880495+0000","last_became_peered":"2024-04-25T13:24:11.880495+0000","last_unstale":"2024-04-25T13:25:07.535087+0000","last_undegraded":"2024-04-25T13:25:07.535087+0000","last_fullsized":"2024-04-25T13:25:07.535087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.031326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530921+0000","last_change":"2024-04-25T13:24:10.854984+0000","last_active":"2024-04-25T13:25:07.530921+0000","last_peered":"2024-04-25T13:25:07.530921+0000","last_clean":"2024-04-25T13:25:07.530921+0000","last_became_active":"2024-04-25T13:24:10.854691+0000","last_became_peered":"2024-04-25T13:24:10.854691+0000","last_unstale":"2024-04-25T13:25:07.530921+0000","last_undegraded":"2024-04-25T13:25:07.530921+0000","last_fullsized":"2024-04-25T13:25:07.530921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:55.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534601+0000","last_change":"2024-04-25T13:24:11.881329+0000","last_active":"2024-04-25T13:25:07.534601+0000","last_peered":"2024-04-25T13:25:07.534601+0000","last_clean":"2024-04-25T13:25:07.534601+0000","last_became_active":"2024-04-25T13:24:11.880958+0000","last_became_peered":"2024-04-25T13:24:11.880958+0000","last_unstale":"2024-04-25T13:25:07.534601+0000","last_undegraded":"2024-04-25T13:25:07.534601+0000","last_fullsized":"2024-04-25T13:25:07.534601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:54.720129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561953+0000","last_change":"2024-04-25T13:24:10.861501+0000","last_active":"2024-04-25T13:25:07.561953+0000","last_peered":"2024-04-25T13:25:07.561953+0000","last_clean":"2024-04-25T13:25:07.561953+0000","last_became_active":"2024-04-25T13:24:10.861155+0000","last_became_peered":"2024-04-25T13:24:10.861155+0000","last_unstale":"2024-04-25T13:25:07.561953+0000","last_undegraded":"2024-04-25T13:25:07.561953+0000","last_fullsized":"2024-04-25T13:25:07.561953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:17.306214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535372+0000","last_change":"2024-04-25T13:24:11.879779+0000","last_active":"2024-04-25T13:25:07.535372+0000","last_peered":"2024-04-25T13:25:07.535372+0000","last_clean":"2024-04-25T13:25:07.535372+0000","last_became_active":"2024-04-25T13:24:11.879285+0000","last_became_peered":"2024-04-25T13:24:11.879285+0000","last_unstale":"2024-04-25T13:25:07.535372+0000","last_undegraded":"2024-04-25T13:25:07.535372+0000","last_fullsized":"2024-04-25T13:25:07.535372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.799094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531172+0000","last_change":"2024-04-25T13:24:11.871059+0000","last_active":"2024-04-25T13:25:07.531172+0000","last_peered":"2024-04-25T13:25:07.531172+0000","last_clean":"2024-04-25T13:25:07.531172+0000","last_became_active":"2024-04-25T13:24:11.869530+0000","last_became_peered":"2024-04-25T13:24:11.869530+0000","last_unstale":"2024-04-25T13:25:07.531172+0000","last_undegraded":"2024-04-25T13:25:07.531172+0000","last_fullsized":"2024-04-25T13:25:07.531172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.095146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528330+0000","last_change":"2024-04-25T13:24:11.876162+0000","last_active":"2024-04-25T13:25:07.528330+0000","last_peered":"2024-04-25T13:25:07.528330+0000","last_clean":"2024-04-25T13:25:07.528330+0000","last_became_active":"2024-04-25T13:24:11.875747+0000","last_became_peered":"2024-04-25T13:24:11.875747+0000","last_unstale":"2024-04-25T13:25:07.528330+0000","last_undegraded":"2024-04-25T13:25:07.528330+0000","last_fullsized":"2024-04-25T13:25:07.528330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:07.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528244+0000","last_change":"2024-04-25T13:24:11.887477+0000","last_active":"2024-04-25T13:25:07.528244+0000","last_peered":"2024-04-25T13:25:07.528244+0000","last_clean":"2024-04-25T13:25:07.528244+0000","last_became_active":"2024-04-25T13:24:11.886951+0000","last_became_peered":"2024-04-25T13:24:11.886951+0000","last_unstale":"2024-04-25T13:25:07.528244+0000","last_undegraded":"2024-04-25T13:25:07.528244+0000","last_fullsized":"2024-04-25T13:25:07.528244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:28.865817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.563315+0000","last_change":"2024-04-25T13:25:07.563315+0000","last_active":"2024-04-25T13:25:07.563315+0000","last_peered":"2024-04-25T13:25:07.563315+0000","last_clean":"2024-04-25T13:25:07.563315+0000","last_became_active":"2024-04-25T13:25:07.562898+0000","last_became_peered":"2024-04-25T13:25:07.562898+0000","last_unstale":"2024-04-25T13:25:07.563315+0000","last_undegraded":"2024-04-25T13:25:07.563315+0000","last_fullsized":"2024-04-25T13:25:07.563315+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:55.921923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561859+0000","last_change":"2024-04-25T13:24:11.876929+0000","last_active":"2024-04-25T13:25:07.561859+0000","last_peered":"2024-04-25T13:25:07.561859+0000","last_clean":"2024-04-25T13:25:07.561859+0000","last_became_active":"2024-04-25T13:24:11.876578+0000","last_became_peered":"2024-04-25T13:24:11.876578+0000","last_unstale":"2024-04-25T13:25:07.561859+0000","last_undegraded":"2024-04-25T13:25:07.561859+0000","last_fullsized":"2024-04-25T13:25:07.561859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.621282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537307+0000","last_change":"2024-04-25T13:25:07.537307+0000","last_active":"2024-04-25T13:25:07.537307+0000","last_peered":"2024-04-25T13:25:07.537307+0000","last_clean":"2024-04-25T13:25:07.537307+0000","last_became_active":"2024-04-25T13:25:07.536757+0000","last_became_peered":"2024-04-25T13:25:07.536757+0000","last_unstale":"2024-04-25T13:25:07.537307+0000","last_undegraded":"2024-04-25T13:25:07.537307+0000","last_fullsized":"2024-04-25T13:25:07.537307+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:26.143315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528815+0000","last_change":"2024-04-25T13:24:11.874008+0000","last_active":"2024-04-25T13:25:07.528815+0000","last_peered":"2024-04-25T13:25:07.528815+0000","last_clean":"2024-04-25T13:25:07.528815+0000","last_became_active":"2024-04-25T13:24:11.873576+0000","last_became_peered":"2024-04-25T13:24:11.873576+0000","last_unstale":"2024-04-25T13:25:07.528815+0000","last_undegraded":"2024-04-25T13:25:07.528815+0000","last_fullsized":"2024-04-25T13:25:07.528815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:14.395329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537492+0000","last_change":"2024-04-25T13:25:07.537492+0000","last_active":"2024-04-25T13:25:07.537492+0000","last_peered":"2024-04-25T13:25:07.537492+0000","last_clean":"2024-04-25T13:25:07.537492+0000","last_became_active":"2024-04-25T13:25:07.537115+0000","last_became_peered":"2024-04-25T13:25:07.537115+0000","last_unstale":"2024-04-25T13:25:07.537492+0000","last_undegraded":"2024-04-25T13:25:07.537492+0000","last_fullsized":"2024-04-25T13:25:07.537492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:39.084520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530375+0000","last_change":"2024-04-25T13:24:11.885922+0000","last_active":"2024-04-25T13:25:07.530375+0000","last_peered":"2024-04-25T13:25:07.530375+0000","last_clean":"2024-04-25T13:25:07.530375+0000","last_became_active":"2024-04-25T13:24:11.885611+0000","last_became_peered":"2024-04-25T13:24:11.885611+0000","last_unstale":"2024-04-25T13:25:07.530375+0000","last_undegraded":"2024-04-25T13:25:07.530375+0000","last_fullsized":"2024-04-25T13:25:07.530375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532015+0000","last_change":"2024-04-25T13:24:11.877912+0000","last_active":"2024-04-25T13:25:07.532015+0000","last_peered":"2024-04-25T13:25:07.532015+0000","last_clean":"2024-04-25T13:25:07.532015+0000","last_became_active":"2024-04-25T13:24:11.877139+0000","last_became_peered":"2024-04-25T13:24:11.877139+0000","last_unstale":"2024-04-25T13:25:07.532015+0000","last_undegraded":"2024-04-25T13:25:07.532015+0000","last_fullsized":"2024-04-25T13:25:07.532015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:45.618556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530656+0000","last_change":"2024-04-25T13:24:11.889267+0000","last_active":"2024-04-25T13:25:07.530656+0000","last_peered":"2024-04-25T13:25:07.530656+0000","last_clean":"2024-04-25T13:25:07.530656+0000","last_became_active":"2024-04-25T13:24:11.888875+0000","last_became_peered":"2024-04-25T13:24:11.888875+0000","last_unstale":"2024-04-25T13:25:07.530656+0000","last_undegraded":"2024-04-25T13:25:07.530656+0000","last_fullsized":"2024-04-25T13:25:07.530656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:41.871075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562081+0000","last_change":"2024-04-25T13:24:11.890664+0000","last_active":"2024-04-25T13:25:07.562081+0000","last_peered":"2024-04-25T13:25:07.562081+0000","last_clean":"2024-04-25T13:25:07.562081+0000","last_became_active":"2024-04-25T13:24:11.890124+0000","last_became_peered":"2024-04-25T13:24:11.890124+0000","last_unstale":"2024-04-25T13:25:07.562081+0000","last_undegraded":"2024-04-25T13:25:07.562081+0000","last_fullsized":"2024-04-25T13:25:07.562081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:55.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531595+0000","last_change":"2024-04-25T13:24:11.876201+0000","last_active":"2024-04-25T13:25:07.531595+0000","last_peered":"2024-04-25T13:25:07.531595+0000","last_clean":"2024-04-25T13:25:07.531595+0000","last_became_active":"2024-04-25T13:24:11.875783+0000","last_became_peered":"2024-04-25T13:24:11.875783+0000","last_unstale":"2024-04-25T13:25:07.531595+0000","last_undegraded":"2024-04-25T13:25:07.531595+0000","last_fullsized":"2024-04-25T13:25:07.531595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.264576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":76,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60532,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":183990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":473642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1124853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":474224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939693,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":474246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234034,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1064709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":473759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":534174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":473732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":473676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117061,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":473980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":534196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1597440,"data_stored":1064309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:25:28.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:29.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:28 smithi069 ceph-mon[32534]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:25:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:28 smithi044 ceph-mon[32590]: from='mgr.14152 172.21.15.37:0/3446868426' entity='mgr.x' 2024-04-25T13:25:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.788+0000 7f747c0cf700 1 -- 172.21.15.37:0/2779443356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101dc0 msgr2=0x7f747410f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.788+0000 7f747c0cf700 1 --2- 172.21.15.37:0/2779443356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101dc0 0x7f747410f830 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f746c00b0a0 tx=0x7f746c00b3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 -- 172.21.15.37:0/2779443356 shutdown_connections 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 --2- 172.21.15.37:0/2779443356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101dc0 0x7f747410f830 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 --2- 172.21.15.37:0/2779443356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 0x7f7474107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 --2- 172.21.15.37:0/2779443356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101400 0x7f7474101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 -- 172.21.15.37:0/2779443356 >> 172.21.15.37:0/2779443356 conn(0x7f7474076c60 msgr2=0x7f7474077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 -- 172.21.15.37:0/2779443356 shutdown_connections 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 -- 172.21.15.37:0/2779443356 wait complete. 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 Processor -- start 2024-04-25T13:25:30.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.789+0000 7f747c0cf700 1 -- start start 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 0x7f7474199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 0x7f747419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7474112d30 con 0x7f7474101400 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7474112bb0 con 0x7f7474101dc0 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7474112eb0 con 0x7f7474107b20 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51424/0 (socket says 172.21.15.37:51424) 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 -- 172.21.15.37:0/572107848 learned_addr learned my addr 172.21.15.37:0/572107848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747966a700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 0x7f7474199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 -- 172.21.15.37:0/572107848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 msgr2=0x7f747419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747a66c700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 0x7f747419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 0x7f747419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 -- 172.21.15.37:0/572107848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 msgr2=0x7f7474199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 0x7f7474199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 -- 172.21.15.37:0/572107848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747419e1f0 con 0x7f7474101400 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747966a700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 0x7f7474199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f7479e6b700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747000b500 tx=0x7f747000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470015020 con 0x7f7474101400 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747419e4e0 con 0x7f7474101400 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7470011ce0 con 0x7f7474101400 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470010d60 con 0x7f7474101400 2024-04-25T13:25:30.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.790+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74741a5d60 con 0x7f7474101400 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.791+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7474069730 con 0x7f7474101400 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.793+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7470018020 con 0x7f7474101400 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.793+0000 7f746affd700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 0x7f74600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.793+0000 7f747966a700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 0x7f74600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.793+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f74700d0250 con 0x7f7474101400 2024-04-25T13:25:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.794+0000 7f747966a700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 0x7f74600a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74740771d0 tx=0x7f7464008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:30.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:30.797+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f747001f020 con 0x7f7474101400 2024-04-25T13:25:30.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:30 smithi037 ceph-mon[29204]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.001+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7474062e80 con 0x7f74600a3cd0 2024-04-25T13:25:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.008+0000 7f746affd700 1 -- 172.21.15.37:0/572107848 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258650 (secure 0 0 0) 0x7f7474062e80 con 0x7f74600a3cd0 2024-04-25T13:25:31.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 msgr2=0x7f74600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 0x7f74600a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74740771d0 tx=0x7f7464008040 comp rx=0 tx=0).stop 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 msgr2=0x7f74741991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747000b500 tx=0x7f747000b8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 shutdown_connections 2024-04-25T13:25:31.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74600a3cd0 0x7f74600a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7474107b20 0x7f747419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7474101400 0x7f74741991e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 --2- 172.21.15.37:0/572107848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7474101dc0 0x7f7474199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 >> 172.21.15.37:0/572107848 conn(0x7f7474076c60 msgr2=0x7f74740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 shutdown_connections 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:31.011+0000 7f747c0cf700 1 -- 172.21.15.37:0/572107848 wait complete. 2024-04-25T13:25:31.015 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T13:25:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:30 smithi069 ceph-mon[32534]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:31.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:30 smithi044 ceph-mon[32590]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:31.715 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-04-25T13:25:29.748514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651932,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":273,"num_read_kb":250,"num_write":259,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1241268,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412744,"kb_used_data":13596,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529080,"statfs":{"total":1151940427776,"available":1151517777920,"internally_reserved":0,"allocated":13922304,"data_stored":7639500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":408532282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002824"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532862+0000","last_change":"2024-04-25T13:24:11.873737+0000","last_active":"2024-04-25T13:25:07.532862+0000","last_peered":"2024-04-25T13:25:07.532862+0000","last_clean":"2024-04-25T13:25:07.532862+0000","last_became_active":"2024-04-25T13:24:11.872598+0000","last_became_peered":"2024-04-25T13:24:11.872598+0000","last_unstale":"2024-04-25T13:25:07.532862+0000","last_undegraded":"2024-04-25T13:25:07.532862+0000","last_fullsized":"2024-04-25T13:25:07.532862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:46.739869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530242+0000","last_change":"2024-04-25T13:24:11.886834+0000","last_active":"2024-04-25T13:25:07.530242+0000","last_peered":"2024-04-25T13:25:07.530242+0000","last_clean":"2024-04-25T13:25:07.530242+0000","last_became_active":"2024-04-25T13:24:11.886460+0000","last_became_peered":"2024-04-25T13:24:11.886460+0000","last_unstale":"2024-04-25T13:25:07.530242+0000","last_undegraded":"2024-04-25T13:25:07.530242+0000","last_fullsized":"2024-04-25T13:25:07.530242+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:45.103106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530814+0000","last_change":"2024-04-25T13:24:11.877582+0000","last_active":"2024-04-25T13:25:07.530814+0000","last_peered":"2024-04-25T13:25:07.530814+0000","last_clean":"2024-04-25T13:25:07.530814+0000","last_became_active":"2024-04-25T13:24:11.877245+0000","last_became_peered":"2024-04-25T13:24:11.877245+0000","last_unstale":"2024-04-25T13:25:07.530814+0000","last_undegraded":"2024-04-25T13:25:07.530814+0000","last_fullsized":"2024-04-25T13:25:07.530814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:05.065255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529356+0000","last_change":"2024-04-25T13:24:11.873522+0000","last_active":"2024-04-25T13:25:07.529356+0000","last_peered":"2024-04-25T13:25:07.529356+0000","last_clean":"2024-04-25T13:25:07.529356+0000","last_became_active":"2024-04-25T13:24:11.872505+0000","last_became_peered":"2024-04-25T13:24:11.872505+0000","last_unstale":"2024-04-25T13:25:07.529356+0000","last_undegraded":"2024-04-25T13:25:07.529356+0000","last_fullsized":"2024-04-25T13:25:07.529356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:15.968935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535050+0000","last_change":"2024-04-25T13:24:11.873353+0000","last_active":"2024-04-25T13:25:07.535050+0000","last_peered":"2024-04-25T13:25:07.535050+0000","last_clean":"2024-04-25T13:25:07.535050+0000","last_became_active":"2024-04-25T13:24:11.872997+0000","last_became_peered":"2024-04-25T13:24:11.872997+0000","last_unstale":"2024-04-25T13:25:07.535050+0000","last_undegraded":"2024-04-25T13:25:07.535050+0000","last_fullsized":"2024-04-25T13:25:07.535050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:15.205105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531291+0000","last_change":"2024-04-25T13:24:11.875994+0000","last_active":"2024-04-25T13:25:07.531291+0000","last_peered":"2024-04-25T13:25:07.531291+0000","last_clean":"2024-04-25T13:25:07.531291+0000","last_became_active":"2024-04-25T13:24:11.875658+0000","last_became_peered":"2024-04-25T13:24:11.875658+0000","last_unstale":"2024-04-25T13:25:07.531291+0000","last_undegraded":"2024-04-25T13:25:07.531291+0000","last_fullsized":"2024-04-25T13:25:07.531291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:20.959715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534078+0000","last_change":"2024-04-25T13:24:11.879417+0000","last_active":"2024-04-25T13:25:07.534078+0000","last_peered":"2024-04-25T13:25:07.534078+0000","last_clean":"2024-04-25T13:25:07.534078+0000","last_became_active":"2024-04-25T13:24:11.879112+0000","last_became_peered":"2024-04-25T13:24:11.879112+0000","last_unstale":"2024-04-25T13:25:07.534078+0000","last_undegraded":"2024-04-25T13:25:07.534078+0000","last_fullsized":"2024-04-25T13:25:07.534078+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:25.250253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531632+0000","last_change":"2024-04-25T13:24:11.883661+0000","last_active":"2024-04-25T13:25:07.531632+0000","last_peered":"2024-04-25T13:25:07.531632+0000","last_clean":"2024-04-25T13:25:07.531632+0000","last_became_active":"2024-04-25T13:24:11.883219+0000","last_became_peered":"2024-04-25T13:24:11.883219+0000","last_unstale":"2024-04-25T13:25:07.531632+0000","last_undegraded":"2024-04-25T13:25:07.531632+0000","last_fullsized":"2024-04-25T13:25:07.531632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:15.731727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530817+0000","last_change":"2024-04-25T13:24:11.880350+0000","last_active":"2024-04-25T13:25:07.530817+0000","last_peered":"2024-04-25T13:25:07.530817+0000","last_clean":"2024-04-25T13:25:07.530817+0000","last_became_active":"2024-04-25T13:24:11.880028+0000","last_became_peered":"2024-04-25T13:24:11.880028+0000","last_unstale":"2024-04-25T13:25:07.530817+0000","last_undegraded":"2024-04-25T13:25:07.530817+0000","last_fullsized":"2024-04-25T13:25:07.530817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:54.832380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531814+0000","last_change":"2024-04-25T13:24:11.879180+0000","last_active":"2024-04-25T13:25:07.531814+0000","last_peered":"2024-04-25T13:25:07.531814+0000","last_clean":"2024-04-25T13:25:07.531814+0000","last_became_active":"2024-04-25T13:24:11.877612+0000","last_became_peered":"2024-04-25T13:24:11.877612+0000","last_unstale":"2024-04-25T13:25:07.531814+0000","last_undegraded":"2024-04-25T13:25:07.531814+0000","last_fullsized":"2024-04-25T13:25:07.531814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:57.269957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535005+0000","last_change":"2024-04-25T13:24:11.880768+0000","last_active":"2024-04-25T13:25:07.535005+0000","last_peered":"2024-04-25T13:25:07.535005+0000","last_clean":"2024-04-25T13:25:07.535005+0000","last_became_active":"2024-04-25T13:24:11.879900+0000","last_became_peered":"2024-04-25T13:24:11.879900+0000","last_unstale":"2024-04-25T13:25:07.535005+0000","last_undegraded":"2024-04-25T13:25:07.535005+0000","last_fullsized":"2024-04-25T13:25:07.535005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:11.941761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529327+0000","last_change":"2024-04-25T13:24:11.873868+0000","last_active":"2024-04-25T13:25:07.529327+0000","last_peered":"2024-04-25T13:25:07.529327+0000","last_clean":"2024-04-25T13:25:07.529327+0000","last_became_active":"2024-04-25T13:24:11.872941+0000","last_became_peered":"2024-04-25T13:24:11.872941+0000","last_unstale":"2024-04-25T13:25:07.529327+0000","last_undegraded":"2024-04-25T13:25:07.529327+0000","last_fullsized":"2024-04-25T13:25:07.529327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:15.580750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562046+0000","last_change":"2024-04-25T13:24:11.888054+0000","last_active":"2024-04-25T13:25:07.562046+0000","last_peered":"2024-04-25T13:25:07.562046+0000","last_clean":"2024-04-25T13:25:07.562046+0000","last_became_active":"2024-04-25T13:24:11.887217+0000","last_became_peered":"2024-04-25T13:24:11.887217+0000","last_unstale":"2024-04-25T13:25:07.562046+0000","last_undegraded":"2024-04-25T13:25:07.562046+0000","last_fullsized":"2024-04-25T13:25:07.562046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:08.058645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530575+0000","last_change":"2024-04-25T13:24:11.878049+0000","last_active":"2024-04-25T13:25:07.530575+0000","last_peered":"2024-04-25T13:25:07.530575+0000","last_clean":"2024-04-25T13:25:07.530575+0000","last_became_active":"2024-04-25T13:24:11.876595+0000","last_became_peered":"2024-04-25T13:24:11.876595+0000","last_unstale":"2024-04-25T13:25:07.530575+0000","last_undegraded":"2024-04-25T13:25:07.530575+0000","last_fullsized":"2024-04-25T13:25:07.530575+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:36.126016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530458+0000","last_change":"2024-04-25T13:24:11.890173+0000","last_active":"2024-04-25T13:25:07.530458+0000","last_peered":"2024-04-25T13:25:07.530458+0000","last_clean":"2024-04-25T13:25:07.530458+0000","last_became_active":"2024-04-25T13:24:11.889865+0000","last_became_peered":"2024-04-25T13:24:11.889865+0000","last_unstale":"2024-04-25T13:25:07.530458+0000","last_undegraded":"2024-04-25T13:25:07.530458+0000","last_fullsized":"2024-04-25T13:25:07.530458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:36.809979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530434+0000","last_change":"2024-04-25T13:24:11.888024+0000","last_active":"2024-04-25T13:25:07.530434+0000","last_peered":"2024-04-25T13:25:07.530434+0000","last_clean":"2024-04-25T13:25:07.530434+0000","last_became_active":"2024-04-25T13:24:11.887497+0000","last_became_peered":"2024-04-25T13:24:11.887497+0000","last_unstale":"2024-04-25T13:25:07.530434+0000","last_undegraded":"2024-04-25T13:25:07.530434+0000","last_fullsized":"2024-04-25T13:25:07.530434+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:30.887282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531971+0000","last_change":"2024-04-25T13:24:11.872532+0000","last_active":"2024-04-25T13:25:07.531971+0000","last_peered":"2024-04-25T13:25:07.531971+0000","last_clean":"2024-04-25T13:25:07.531971+0000","last_became_active":"2024-04-25T13:24:11.871047+0000","last_became_peered":"2024-04-25T13:24:11.871047+0000","last_unstale":"2024-04-25T13:25:07.531971+0000","last_undegraded":"2024-04-25T13:25:07.531971+0000","last_fullsized":"2024-04-25T13:25:07.531971+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:21.818593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530231+0000","last_change":"2024-04-25T13:24:11.869003+0000","last_active":"2024-04-25T13:25:07.530231+0000","last_peered":"2024-04-25T13:25:07.530231+0000","last_clean":"2024-04-25T13:25:07.530231+0000","last_became_active":"2024-04-25T13:24:11.868639+0000","last_became_peered":"2024-04-25T13:24:11.868639+0000","last_unstale":"2024-04-25T13:25:07.530231+0000","last_undegraded":"2024-04-25T13:25:07.530231+0000","last_fullsized":"2024-04-25T13:25:07.530231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:03.366661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528630+0000","last_change":"2024-04-25T13:24:11.878947+0000","last_active":"2024-04-25T13:25:07.528630+0000","last_peered":"2024-04-25T13:25:07.528630+0000","last_clean":"2024-04-25T13:25:07.528630+0000","last_became_active":"2024-04-25T13:24:11.878344+0000","last_became_peered":"2024-04-25T13:24:11.878344+0000","last_unstale":"2024-04-25T13:25:07.528630+0000","last_undegraded":"2024-04-25T13:25:07.528630+0000","last_fullsized":"2024-04-25T13:25:07.528630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:54.653873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530784+0000","last_change":"2024-04-25T13:24:10.856180+0000","last_active":"2024-04-25T13:25:07.530784+0000","last_peered":"2024-04-25T13:25:07.530784+0000","last_clean":"2024-04-25T13:25:07.530784+0000","last_became_active":"2024-04-25T13:24:10.855746+0000","last_became_peered":"2024-04-25T13:24:10.855746+0000","last_unstale":"2024-04-25T13:25:07.530784+0000","last_undegraded":"2024-04-25T13:25:07.530784+0000","last_fullsized":"2024-04-25T13:25:07.530784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:58.312713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532741+0000","last_change":"2024-04-25T13:24:11.889286+0000","last_active":"2024-04-25T13:25:07.532741+0000","last_peered":"2024-04-25T13:25:07.532741+0000","last_clean":"2024-04-25T13:25:07.532741+0000","last_became_active":"2024-04-25T13:24:11.888950+0000","last_became_peered":"2024-04-25T13:24:11.888950+0000","last_unstale":"2024-04-25T13:25:07.532741+0000","last_undegraded":"2024-04-25T13:25:07.532741+0000","last_fullsized":"2024-04-25T13:25:07.532741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:31.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.533022+0000","last_change":"2024-04-25T13:24:10.859008+0000","last_active":"2024-04-25T13:25:07.533022+0000","last_peered":"2024-04-25T13:25:07.533022+0000","last_clean":"2024-04-25T13:25:07.533022+0000","last_became_active":"2024-04-25T13:24:10.858350+0000","last_became_peered":"2024-04-25T13:24:10.858350+0000","last_unstale":"2024-04-25T13:25:07.533022+0000","last_undegraded":"2024-04-25T13:25:07.533022+0000","last_fullsized":"2024-04-25T13:25:07.533022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.048592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531248+0000","last_change":"2024-04-25T13:24:11.879185+0000","last_active":"2024-04-25T13:25:07.531248+0000","last_peered":"2024-04-25T13:25:07.531248+0000","last_clean":"2024-04-25T13:25:07.531248+0000","last_became_active":"2024-04-25T13:24:11.878808+0000","last_became_peered":"2024-04-25T13:24:11.878808+0000","last_unstale":"2024-04-25T13:25:07.531248+0000","last_undegraded":"2024-04-25T13:25:07.531248+0000","last_fullsized":"2024-04-25T13:25:07.531248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:58.408529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535898+0000","last_change":"2024-04-25T13:24:10.861742+0000","last_active":"2024-04-25T13:25:07.535898+0000","last_peered":"2024-04-25T13:25:07.535898+0000","last_clean":"2024-04-25T13:25:07.535898+0000","last_became_active":"2024-04-25T13:24:10.861302+0000","last_became_peered":"2024-04-25T13:24:10.861302+0000","last_unstale":"2024-04-25T13:25:07.535898+0000","last_undegraded":"2024-04-25T13:25:07.535898+0000","last_fullsized":"2024-04-25T13:25:07.535898+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.470804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530803+0000","last_change":"2024-04-25T13:24:11.873872+0000","last_active":"2024-04-25T13:25:07.530803+0000","last_peered":"2024-04-25T13:25:07.530803+0000","last_clean":"2024-04-25T13:25:07.530803+0000","last_became_active":"2024-04-25T13:24:11.873341+0000","last_became_peered":"2024-04-25T13:24:11.873341+0000","last_unstale":"2024-04-25T13:25:07.530803+0000","last_undegraded":"2024-04-25T13:25:07.530803+0000","last_fullsized":"2024-04-25T13:25:07.530803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:46.271742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535165+0000","last_change":"2024-04-25T13:24:10.856554+0000","last_active":"2024-04-25T13:25:07.535165+0000","last_peered":"2024-04-25T13:25:07.535165+0000","last_clean":"2024-04-25T13:25:07.535165+0000","last_became_active":"2024-04-25T13:24:10.856254+0000","last_became_peered":"2024-04-25T13:24:10.856254+0000","last_unstale":"2024-04-25T13:25:07.535165+0000","last_undegraded":"2024-04-25T13:25:07.535165+0000","last_fullsized":"2024-04-25T13:25:07.535165+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:15.843985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530992+0000","last_change":"2024-04-25T13:24:11.891970+0000","last_active":"2024-04-25T13:25:07.530992+0000","last_peered":"2024-04-25T13:25:07.530992+0000","last_clean":"2024-04-25T13:25:07.530992+0000","last_became_active":"2024-04-25T13:24:11.891537+0000","last_became_peered":"2024-04-25T13:24:11.891537+0000","last_unstale":"2024-04-25T13:25:07.530992+0000","last_undegraded":"2024-04-25T13:25:07.530992+0000","last_fullsized":"2024-04-25T13:25:07.530992+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.509206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530789+0000","last_change":"2024-04-25T13:24:10.857130+0000","last_active":"2024-04-25T13:25:07.530789+0000","last_peered":"2024-04-25T13:25:07.530789+0000","last_clean":"2024-04-25T13:25:07.530789+0000","last_became_active":"2024-04-25T13:24:10.856747+0000","last_became_peered":"2024-04-25T13:24:10.856747+0000","last_unstale":"2024-04-25T13:25:07.530789+0000","last_undegraded":"2024-04-25T13:25:07.530789+0000","last_fullsized":"2024-04-25T13:25:07.530789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:11.366223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534986+0000","last_change":"2024-04-25T13:24:11.873404+0000","last_active":"2024-04-25T13:25:07.534986+0000","last_peered":"2024-04-25T13:25:07.534986+0000","last_clean":"2024-04-25T13:25:07.534986+0000","last_became_active":"2024-04-25T13:24:11.873027+0000","last_became_peered":"2024-04-25T13:24:11.873027+0000","last_unstale":"2024-04-25T13:25:07.534986+0000","last_undegraded":"2024-04-25T13:25:07.534986+0000","last_fullsized":"2024-04-25T13:25:07.534986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:32.789300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'1","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531172+0000","last_change":"2024-04-25T13:24:10.853602+0000","last_active":"2024-04-25T13:25:07.531172+0000","last_peered":"2024-04-25T13:25:07.531172+0000","last_clean":"2024-04-25T13:25:07.531172+0000","last_became_active":"2024-04-25T13:24:10.853237+0000","last_became_peered":"2024-04-25T13:24:10.853237+0000","last_unstale":"2024-04-25T13:25:07.531172+0000","last_undegraded":"2024-04-25T13:25:07.531172+0000","last_fullsized":"2024-04-25T13:25:07.531172+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.575835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562946+0000","last_change":"2024-04-25T13:25:07.562946+0000","last_active":"2024-04-25T13:25:07.562946+0000","last_peered":"2024-04-25T13:25:07.562946+0000","last_clean":"2024-04-25T13:25:07.562946+0000","last_became_active":"2024-04-25T13:25:07.562651+0000","last_became_peered":"2024-04-25T13:25:07.562651+0000","last_unstale":"2024-04-25T13:25:07.562946+0000","last_undegraded":"2024-04-25T13:25:07.562946+0000","last_fullsized":"2024-04-25T13:25:07.562946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:34.377961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561811+0000","last_change":"2024-04-25T13:24:10.851042+0000","last_active":"2024-04-25T13:25:07.561811+0000","last_peered":"2024-04-25T13:25:07.561811+0000","last_clean":"2024-04-25T13:25:07.561811+0000","last_became_active":"2024-04-25T13:24:10.850619+0000","last_became_peered":"2024-04-25T13:24:10.850619+0000","last_unstale":"2024-04-25T13:25:07.561811+0000","last_undegraded":"2024-04-25T13:25:07.561811+0000","last_fullsized":"2024-04-25T13:25:07.561811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:23.577971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531694+0000","last_change":"2024-04-25T13:24:11.880170+0000","last_active":"2024-04-25T13:25:07.531694+0000","last_peered":"2024-04-25T13:25:07.531694+0000","last_clean":"2024-04-25T13:25:07.531694+0000","last_became_active":"2024-04-25T13:24:11.879802+0000","last_became_peered":"2024-04-25T13:24:11.879802+0000","last_unstale":"2024-04-25T13:25:07.531694+0000","last_undegraded":"2024-04-25T13:25:07.531694+0000","last_fullsized":"2024-04-25T13:25:07.531694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:25.231071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530344+0000","last_change":"2024-04-25T13:24:10.852667+0000","last_active":"2024-04-25T13:25:07.530344+0000","last_peered":"2024-04-25T13:25:07.530344+0000","last_clean":"2024-04-25T13:25:07.530344+0000","last_became_active":"2024-04-25T13:24:10.852222+0000","last_became_peered":"2024-04-25T13:24:10.852222+0000","last_unstale":"2024-04-25T13:25:07.530344+0000","last_undegraded":"2024-04-25T13:25:07.530344+0000","last_fullsized":"2024-04-25T13:25:07.530344+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:06.711152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530027+0000","last_change":"2024-04-25T13:24:11.867984+0000","last_active":"2024-04-25T13:25:07.530027+0000","last_peered":"2024-04-25T13:25:07.530027+0000","last_clean":"2024-04-25T13:25:07.530027+0000","last_became_active":"2024-04-25T13:24:11.867517+0000","last_became_peered":"2024-04-25T13:24:11.867517+0000","last_unstale":"2024-04-25T13:25:07.530027+0000","last_undegraded":"2024-04-25T13:25:07.530027+0000","last_fullsized":"2024-04-25T13:25:07.530027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:41.540573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531860+0000","last_change":"2024-04-25T13:24:10.855925+0000","last_active":"2024-04-25T13:25:07.531860+0000","last_peered":"2024-04-25T13:25:07.531860+0000","last_clean":"2024-04-25T13:25:07.531860+0000","last_became_active":"2024-04-25T13:24:10.855603+0000","last_became_peered":"2024-04-25T13:24:10.855603+0000","last_unstale":"2024-04-25T13:25:07.531860+0000","last_undegraded":"2024-04-25T13:25:07.531860+0000","last_fullsized":"2024-04-25T13:25:07.531860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:26.796979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530912+0000","last_change":"2024-04-25T13:24:11.873683+0000","last_active":"2024-04-25T13:25:07.530912+0000","last_peered":"2024-04-25T13:25:07.530912+0000","last_clean":"2024-04-25T13:25:07.530912+0000","last_became_active":"2024-04-25T13:24:11.873307+0000","last_became_peered":"2024-04-25T13:24:11.873307+0000","last_unstale":"2024-04-25T13:25:07.530912+0000","last_undegraded":"2024-04-25T13:25:07.530912+0000","last_fullsized":"2024-04-25T13:25:07.530912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:44.305349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535141+0000","last_change":"2024-04-25T13:24:10.857124+0000","last_active":"2024-04-25T13:25:07.535141+0000","last_peered":"2024-04-25T13:25:07.535141+0000","last_clean":"2024-04-25T13:25:07.535141+0000","last_became_active":"2024-04-25T13:24:10.856396+0000","last_became_peered":"2024-04-25T13:24:10.856396+0000","last_unstale":"2024-04-25T13:25:07.535141+0000","last_undegraded":"2024-04-25T13:25:07.535141+0000","last_fullsized":"2024-04-25T13:25:07.535141+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:58.502205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531165+0000","last_change":"2024-04-25T13:24:11.882305+0000","last_active":"2024-04-25T13:25:07.531165+0000","last_peered":"2024-04-25T13:25:07.531165+0000","last_clean":"2024-04-25T13:25:07.531165+0000","last_became_active":"2024-04-25T13:24:11.881976+0000","last_became_peered":"2024-04-25T13:24:11.881976+0000","last_unstale":"2024-04-25T13:25:07.531165+0000","last_undegraded":"2024-04-25T13:25:07.531165+0000","last_fullsized":"2024-04-25T13:25:07.531165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:46.254871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'3","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531049+0000","last_change":"2024-04-25T13:24:10.864302+0000","last_active":"2024-04-25T13:25:07.531049+0000","last_peered":"2024-04-25T13:25:07.531049+0000","last_clean":"2024-04-25T13:25:07.531049+0000","last_became_active":"2024-04-25T13:24:10.863955+0000","last_became_peered":"2024-04-25T13:24:10.863955+0000","last_unstale":"2024-04-25T13:25:07.531049+0000","last_undegraded":"2024-04-25T13:25:07.531049+0000","last_fullsized":"2024-04-25T13:25:07.531049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:35.394778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531042+0000","last_change":"2024-04-25T13:24:11.876293+0000","last_active":"2024-04-25T13:25:07.531042+0000","last_peered":"2024-04-25T13:25:07.531042+0000","last_clean":"2024-04-25T13:25:07.531042+0000","last_became_active":"2024-04-25T13:24:11.875927+0000","last_became_peered":"2024-04-25T13:24:11.875927+0000","last_unstale":"2024-04-25T13:25:07.531042+0000","last_undegraded":"2024-04-25T13:25:07.531042+0000","last_fullsized":"2024-04-25T13:25:07.531042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:47.884328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":209,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:24.566420+0000","last_change":"2024-04-25T13:25:07.543419+0000","last_active":"2024-04-25T13:25:24.566420+0000","last_peered":"2024-04-25T13:25:24.566420+0000","last_clean":"2024-04-25T13:25:24.566420+0000","last_became_active":"2024-04-25T13:25:07.535875+0000","last_became_peered":"2024-04-25T13:25:07.535875+0000","last_unstale":"2024-04-25T13:25:24.566420+0000","last_undegraded":"2024-04-25T13:25:24.566420+0000","last_fullsized":"2024-04-25T13:25:24.566420+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:18.213369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60532,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":131,"num_read_kb":125,"num_write":27,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":60532,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535657+0000","last_change":"2024-04-25T13:24:11.886845+0000","last_active":"2024-04-25T13:25:07.535657+0000","last_peered":"2024-04-25T13:25:07.535657+0000","last_clean":"2024-04-25T13:25:07.535657+0000","last_became_active":"2024-04-25T13:24:11.885457+0000","last_became_peered":"2024-04-25T13:24:11.885457+0000","last_unstale":"2024-04-25T13:25:07.535657+0000","last_undegraded":"2024-04-25T13:25:07.535657+0000","last_fullsized":"2024-04-25T13:25:07.535657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:56.134026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528563+0000","last_change":"2024-04-25T13:24:10.865292+0000","last_active":"2024-04-25T13:25:07.528563+0000","last_peered":"2024-04-25T13:25:07.528563+0000","last_clean":"2024-04-25T13:25:07.528563+0000","last_became_active":"2024-04-25T13:24:10.864940+0000","last_became_peered":"2024-04-25T13:24:10.864940+0000","last_unstale":"2024-04-25T13:25:07.528563+0000","last_undegraded":"2024-04-25T13:25:07.528563+0000","last_fullsized":"2024-04-25T13:25:07.528563+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:54.982815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561814+0000","last_change":"2024-04-25T13:24:11.885158+0000","last_active":"2024-04-25T13:25:07.561814+0000","last_peered":"2024-04-25T13:25:07.561814+0000","last_clean":"2024-04-25T13:25:07.561814+0000","last_became_active":"2024-04-25T13:24:11.884367+0000","last_became_peered":"2024-04-25T13:24:11.884367+0000","last_unstale":"2024-04-25T13:25:07.561814+0000","last_undegraded":"2024-04-25T13:25:07.561814+0000","last_fullsized":"2024-04-25T13:25:07.561814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:48.244922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530650+0000","last_change":"2024-04-25T13:24:10.854269+0000","last_active":"2024-04-25T13:25:07.530650+0000","last_peered":"2024-04-25T13:25:07.530650+0000","last_clean":"2024-04-25T13:25:07.530650+0000","last_became_active":"2024-04-25T13:24:10.853968+0000","last_became_peered":"2024-04-25T13:24:10.853968+0000","last_unstale":"2024-04-25T13:25:07.530650+0000","last_undegraded":"2024-04-25T13:25:07.530650+0000","last_fullsized":"2024-04-25T13:25:07.530650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:42.771743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534934+0000","last_change":"2024-04-25T13:24:11.887468+0000","last_active":"2024-04-25T13:25:07.534934+0000","last_peered":"2024-04-25T13:25:07.534934+0000","last_clean":"2024-04-25T13:25:07.534934+0000","last_became_active":"2024-04-25T13:24:11.887097+0000","last_became_peered":"2024-04-25T13:24:11.887097+0000","last_unstale":"2024-04-25T13:25:07.534934+0000","last_undegraded":"2024-04-25T13:25:07.534934+0000","last_fullsized":"2024-04-25T13:25:07.534934+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:23.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528632+0000","last_change":"2024-04-25T13:24:10.854484+0000","last_active":"2024-04-25T13:25:07.528632+0000","last_peered":"2024-04-25T13:25:07.528632+0000","last_clean":"2024-04-25T13:25:07.528632+0000","last_became_active":"2024-04-25T13:24:10.854152+0000","last_became_peered":"2024-04-25T13:24:10.854152+0000","last_unstale":"2024-04-25T13:25:07.528632+0000","last_undegraded":"2024-04-25T13:25:07.528632+0000","last_fullsized":"2024-04-25T13:25:07.528632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:25.940414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534026+0000","last_change":"2024-04-25T13:24:11.880661+0000","last_active":"2024-04-25T13:25:07.534026+0000","last_peered":"2024-04-25T13:25:07.534026+0000","last_clean":"2024-04-25T13:25:07.534026+0000","last_became_active":"2024-04-25T13:24:11.880099+0000","last_became_peered":"2024-04-25T13:24:11.880099+0000","last_unstale":"2024-04-25T13:25:07.534026+0000","last_undegraded":"2024-04-25T13:25:07.534026+0000","last_fullsized":"2024-04-25T13:25:07.534026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:35.334583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534443+0000","last_change":"2024-04-25T13:24:10.863356+0000","last_active":"2024-04-25T13:25:07.534443+0000","last_peered":"2024-04-25T13:25:07.534443+0000","last_clean":"2024-04-25T13:25:07.534443+0000","last_became_active":"2024-04-25T13:24:10.863024+0000","last_became_peered":"2024-04-25T13:24:10.863024+0000","last_unstale":"2024-04-25T13:25:07.534443+0000","last_undegraded":"2024-04-25T13:25:07.534443+0000","last_fullsized":"2024-04-25T13:25:07.534443+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:11.520794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530444+0000","last_change":"2024-04-25T13:24:11.891787+0000","last_active":"2024-04-25T13:25:07.530444+0000","last_peered":"2024-04-25T13:25:07.530444+0000","last_clean":"2024-04-25T13:25:07.530444+0000","last_became_active":"2024-04-25T13:24:11.891466+0000","last_became_peered":"2024-04-25T13:24:11.891466+0000","last_unstale":"2024-04-25T13:25:07.530444+0000","last_undegraded":"2024-04-25T13:25:07.530444+0000","last_fullsized":"2024-04-25T13:25:07.530444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:11.200558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535661+0000","last_change":"2024-04-25T13:24:10.852320+0000","last_active":"2024-04-25T13:25:07.535661+0000","last_peered":"2024-04-25T13:25:07.535661+0000","last_clean":"2024-04-25T13:25:07.535661+0000","last_became_active":"2024-04-25T13:24:10.852007+0000","last_became_peered":"2024-04-25T13:24:10.852007+0000","last_unstale":"2024-04-25T13:25:07.535661+0000","last_undegraded":"2024-04-25T13:25:07.535661+0000","last_fullsized":"2024-04-25T13:25:07.535661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.582461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534523+0000","last_change":"2024-04-25T13:25:07.534523+0000","last_active":"2024-04-25T13:25:07.534523+0000","last_peered":"2024-04-25T13:25:07.534523+0000","last_clean":"2024-04-25T13:25:07.534523+0000","last_became_active":"2024-04-25T13:25:07.534117+0000","last_became_peered":"2024-04-25T13:25:07.534117+0000","last_unstale":"2024-04-25T13:25:07.534523+0000","last_undegraded":"2024-04-25T13:25:07.534523+0000","last_fullsized":"2024-04-25T13:25:07.534523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:12.904043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530081+0000","last_change":"2024-04-25T13:24:10.857715+0000","last_active":"2024-04-25T13:25:07.530081+0000","last_peered":"2024-04-25T13:25:07.530081+0000","last_clean":"2024-04-25T13:25:07.530081+0000","last_became_active":"2024-04-25T13:24:10.857398+0000","last_became_peered":"2024-04-25T13:24:10.857398+0000","last_unstale":"2024-04-25T13:25:07.530081+0000","last_undegraded":"2024-04-25T13:25:07.530081+0000","last_fullsized":"2024-04-25T13:25:07.530081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:58.189652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531704+0000","last_change":"2024-04-25T13:24:11.875065+0000","last_active":"2024-04-25T13:25:07.531704+0000","last_peered":"2024-04-25T13:25:07.531704+0000","last_clean":"2024-04-25T13:25:07.531704+0000","last_became_active":"2024-04-25T13:24:11.874433+0000","last_became_peered":"2024-04-25T13:24:11.874433+0000","last_unstale":"2024-04-25T13:25:07.531704+0000","last_undegraded":"2024-04-25T13:25:07.531704+0000","last_fullsized":"2024-04-25T13:25:07.531704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:07.430007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534227+0000","last_change":"2024-04-25T13:24:10.850250+0000","last_active":"2024-04-25T13:25:07.534227+0000","last_peered":"2024-04-25T13:25:07.534227+0000","last_clean":"2024-04-25T13:25:07.534227+0000","last_became_active":"2024-04-25T13:24:10.849773+0000","last_became_peered":"2024-04-25T13:24:10.849773+0000","last_unstale":"2024-04-25T13:25:07.534227+0000","last_undegraded":"2024-04-25T13:25:07.534227+0000","last_fullsized":"2024-04-25T13:25:07.534227+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:59.609421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531693+0000","last_change":"2024-04-25T13:24:11.878625+0000","last_active":"2024-04-25T13:25:07.531693+0000","last_peered":"2024-04-25T13:25:07.531693+0000","last_clean":"2024-04-25T13:25:07.531693+0000","last_became_active":"2024-04-25T13:24:11.878311+0000","last_became_peered":"2024-04-25T13:24:11.878311+0000","last_unstale":"2024-04-25T13:25:07.531693+0000","last_undegraded":"2024-04-25T13:25:07.531693+0000","last_fullsized":"2024-04-25T13:25:07.531693+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:48.363441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530532+0000","last_change":"2024-04-25T13:24:11.887874+0000","last_active":"2024-04-25T13:25:07.530532+0000","last_peered":"2024-04-25T13:25:07.530532+0000","last_clean":"2024-04-25T13:25:07.530532+0000","last_became_active":"2024-04-25T13:24:11.887558+0000","last_became_peered":"2024-04-25T13:24:11.887558+0000","last_unstale":"2024-04-25T13:25:07.530532+0000","last_undegraded":"2024-04-25T13:25:07.530532+0000","last_fullsized":"2024-04-25T13:25:07.530532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:49.861447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561681+0000","last_change":"2024-04-25T13:24:10.859643+0000","last_active":"2024-04-25T13:25:07.561681+0000","last_peered":"2024-04-25T13:25:07.561681+0000","last_clean":"2024-04-25T13:25:07.561681+0000","last_became_active":"2024-04-25T13:24:10.859312+0000","last_became_peered":"2024-04-25T13:24:10.859312+0000","last_unstale":"2024-04-25T13:25:07.561681+0000","last_undegraded":"2024-04-25T13:25:07.561681+0000","last_fullsized":"2024-04-25T13:25:07.561681+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.513203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532592+0000","last_change":"2024-04-25T13:24:11.883821+0000","last_active":"2024-04-25T13:25:07.532592+0000","last_peered":"2024-04-25T13:25:07.532592+0000","last_clean":"2024-04-25T13:25:07.532592+0000","last_became_active":"2024-04-25T13:24:11.883063+0000","last_became_peered":"2024-04-25T13:24:11.883063+0000","last_unstale":"2024-04-25T13:25:07.532592+0000","last_undegraded":"2024-04-25T13:25:07.532592+0000","last_fullsized":"2024-04-25T13:25:07.532592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:39.845694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529094+0000","last_change":"2024-04-25T13:24:10.851525+0000","last_active":"2024-04-25T13:25:07.529094+0000","last_peered":"2024-04-25T13:25:07.529094+0000","last_clean":"2024-04-25T13:25:07.529094+0000","last_became_active":"2024-04-25T13:24:10.851171+0000","last_became_peered":"2024-04-25T13:24:10.851171+0000","last_unstale":"2024-04-25T13:25:07.529094+0000","last_undegraded":"2024-04-25T13:25:07.529094+0000","last_fullsized":"2024-04-25T13:25:07.529094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534069+0000","last_change":"2024-04-25T13:22:58.125848+0000","last_active":"2024-04-25T13:25:07.534069+0000","last_peered":"2024-04-25T13:25:07.534069+0000","last_clean":"2024-04-25T13:25:07.534069+0000","last_became_active":"2024-04-25T13:22:58.125374+0000","last_became_peered":"2024-04-25T13:22:58.125374+0000","last_unstale":"2024-04-25T13:25:07.534069+0000","last_undegraded":"2024-04-25T13:25:07.534069+0000","last_fullsized":"2024-04-25T13:25:07.534069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:19:36.251667+0000","last_clean_scrub_stamp":"2024-04-25T13:19:36.251667+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.304605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561679+0000","last_change":"2024-04-25T13:24:11.883648+0000","last_active":"2024-04-25T13:25:07.561679+0000","last_peered":"2024-04-25T13:25:07.561679+0000","last_clean":"2024-04-25T13:25:07.561679+0000","last_became_active":"2024-04-25T13:24:11.879538+0000","last_became_peered":"2024-04-25T13:24:11.879538+0000","last_unstale":"2024-04-25T13:25:07.561679+0000","last_undegraded":"2024-04-25T13:25:07.561679+0000","last_fullsized":"2024-04-25T13:25:07.561679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:11.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537012+0000","last_change":"2024-04-25T13:25:07.537012+0000","last_active":"2024-04-25T13:25:07.537012+0000","last_peered":"2024-04-25T13:25:07.537012+0000","last_clean":"2024-04-25T13:25:07.537012+0000","last_became_active":"2024-04-25T13:25:07.536537+0000","last_became_peered":"2024-04-25T13:25:07.536537+0000","last_unstale":"2024-04-25T13:25:07.537012+0000","last_undegraded":"2024-04-25T13:25:07.537012+0000","last_fullsized":"2024-04-25T13:25:07.537012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:33.420293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535778+0000","last_change":"2024-04-25T13:24:10.864221+0000","last_active":"2024-04-25T13:25:07.535778+0000","last_peered":"2024-04-25T13:25:07.535778+0000","last_clean":"2024-04-25T13:25:07.535778+0000","last_became_active":"2024-04-25T13:24:10.863892+0000","last_became_peered":"2024-04-25T13:24:10.863892+0000","last_unstale":"2024-04-25T13:25:07.535778+0000","last_undegraded":"2024-04-25T13:25:07.535778+0000","last_fullsized":"2024-04-25T13:25:07.535778+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:51.229865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561995+0000","last_change":"2024-04-25T13:24:11.878257+0000","last_active":"2024-04-25T13:25:07.561995+0000","last_peered":"2024-04-25T13:25:07.561995+0000","last_clean":"2024-04-25T13:25:07.561995+0000","last_became_active":"2024-04-25T13:24:11.877926+0000","last_became_peered":"2024-04-25T13:24:11.877926+0000","last_unstale":"2024-04-25T13:25:07.561995+0000","last_undegraded":"2024-04-25T13:25:07.561995+0000","last_fullsized":"2024-04-25T13:25:07.561995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:32.873011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.539670+0000","last_change":"2024-04-25T13:25:07.539670+0000","last_active":"2024-04-25T13:25:07.539670+0000","last_peered":"2024-04-25T13:25:07.539670+0000","last_clean":"2024-04-25T13:25:07.539670+0000","last_became_active":"2024-04-25T13:25:07.535378+0000","last_became_peered":"2024-04-25T13:25:07.535378+0000","last_unstale":"2024-04-25T13:25:07.539670+0000","last_undegraded":"2024-04-25T13:25:07.539670+0000","last_fullsized":"2024-04-25T13:25:07.539670+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:31.843073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.536213+0000","last_change":"2024-04-25T13:25:07.535542+0000","last_active":"2024-04-25T13:25:07.536213+0000","last_peered":"2024-04-25T13:25:07.536213+0000","last_clean":"2024-04-25T13:25:07.536213+0000","last_became_active":"2024-04-25T13:25:07.535142+0000","last_became_peered":"2024-04-25T13:25:07.535142+0000","last_unstale":"2024-04-25T13:25:07.536213+0000","last_undegraded":"2024-04-25T13:25:07.536213+0000","last_fullsized":"2024-04-25T13:25:07.536213+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:30.618062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.529311+0000","last_change":"2024-04-25T13:24:10.854441+0000","last_active":"2024-04-25T13:25:07.529311+0000","last_peered":"2024-04-25T13:25:07.529311+0000","last_clean":"2024-04-25T13:25:07.529311+0000","last_became_active":"2024-04-25T13:24:10.854086+0000","last_became_peered":"2024-04-25T13:24:10.854086+0000","last_unstale":"2024-04-25T13:25:07.529311+0000","last_undegraded":"2024-04-25T13:25:07.529311+0000","last_fullsized":"2024-04-25T13:25:07.529311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:17.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530861+0000","last_change":"2024-04-25T13:24:11.880494+0000","last_active":"2024-04-25T13:25:07.530861+0000","last_peered":"2024-04-25T13:25:07.530861+0000","last_clean":"2024-04-25T13:25:07.530861+0000","last_became_active":"2024-04-25T13:24:11.880074+0000","last_became_peered":"2024-04-25T13:24:11.880074+0000","last_unstale":"2024-04-25T13:25:07.530861+0000","last_undegraded":"2024-04-25T13:25:07.530861+0000","last_fullsized":"2024-04-25T13:25:07.530861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:50.993165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535837+0000","last_change":"2024-04-25T13:25:07.535837+0000","last_active":"2024-04-25T13:25:07.535837+0000","last_peered":"2024-04-25T13:25:07.535837+0000","last_clean":"2024-04-25T13:25:07.535837+0000","last_became_active":"2024-04-25T13:25:07.535415+0000","last_became_peered":"2024-04-25T13:25:07.535415+0000","last_unstale":"2024-04-25T13:25:07.535837+0000","last_undegraded":"2024-04-25T13:25:07.535837+0000","last_fullsized":"2024-04-25T13:25:07.535837+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:25.847092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534583+0000","last_change":"2024-04-25T13:24:11.878770+0000","last_active":"2024-04-25T13:25:07.534583+0000","last_peered":"2024-04-25T13:25:07.534583+0000","last_clean":"2024-04-25T13:25:07.534583+0000","last_became_active":"2024-04-25T13:24:11.878424+0000","last_became_peered":"2024-04-25T13:24:11.878424+0000","last_unstale":"2024-04-25T13:25:07.534583+0000","last_undegraded":"2024-04-25T13:25:07.534583+0000","last_fullsized":"2024-04-25T13:25:07.534583+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:25.016200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531910+0000","last_change":"2024-04-25T13:24:10.857246+0000","last_active":"2024-04-25T13:25:07.531910+0000","last_peered":"2024-04-25T13:25:07.531910+0000","last_clean":"2024-04-25T13:25:07.531910+0000","last_became_active":"2024-04-25T13:24:10.856922+0000","last_became_peered":"2024-04-25T13:24:10.856922+0000","last_unstale":"2024-04-25T13:25:07.531910+0000","last_undegraded":"2024-04-25T13:25:07.531910+0000","last_fullsized":"2024-04-25T13:25:07.531910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:08.039339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531105+0000","last_change":"2024-04-25T13:24:11.873225+0000","last_active":"2024-04-25T13:25:07.531105+0000","last_peered":"2024-04-25T13:25:07.531105+0000","last_clean":"2024-04-25T13:25:07.531105+0000","last_became_active":"2024-04-25T13:24:11.872855+0000","last_became_peered":"2024-04-25T13:24:11.872855+0000","last_unstale":"2024-04-25T13:25:07.531105+0000","last_undegraded":"2024-04-25T13:25:07.531105+0000","last_fullsized":"2024-04-25T13:25:07.531105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:31.523621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528331+0000","last_change":"2024-04-25T13:24:10.863165+0000","last_active":"2024-04-25T13:25:07.528331+0000","last_peered":"2024-04-25T13:25:07.528331+0000","last_clean":"2024-04-25T13:25:07.528331+0000","last_became_active":"2024-04-25T13:24:10.862596+0000","last_became_peered":"2024-04-25T13:24:10.862596+0000","last_unstale":"2024-04-25T13:25:07.528331+0000","last_undegraded":"2024-04-25T13:25:07.528331+0000","last_fullsized":"2024-04-25T13:25:07.528331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:51.414158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532355+0000","last_change":"2024-04-25T13:24:11.885852+0000","last_active":"2024-04-25T13:25:07.532355+0000","last_peered":"2024-04-25T13:25:07.532355+0000","last_clean":"2024-04-25T13:25:07.532355+0000","last_became_active":"2024-04-25T13:24:11.885508+0000","last_became_peered":"2024-04-25T13:24:11.885508+0000","last_unstale":"2024-04-25T13:25:07.532355+0000","last_undegraded":"2024-04-25T13:25:07.532355+0000","last_fullsized":"2024-04-25T13:25:07.532355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:41.575351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535089+0000","last_change":"2024-04-25T13:24:10.861701+0000","last_active":"2024-04-25T13:25:07.535089+0000","last_peered":"2024-04-25T13:25:07.535089+0000","last_clean":"2024-04-25T13:25:07.535089+0000","last_became_active":"2024-04-25T13:24:10.861356+0000","last_became_peered":"2024-04-25T13:24:10.861356+0000","last_unstale":"2024-04-25T13:25:07.535089+0000","last_undegraded":"2024-04-25T13:25:07.535089+0000","last_fullsized":"2024-04-25T13:25:07.535089+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:24.300835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534372+0000","last_change":"2024-04-25T13:24:11.883977+0000","last_active":"2024-04-25T13:25:07.534372+0000","last_peered":"2024-04-25T13:25:07.534372+0000","last_clean":"2024-04-25T13:25:07.534372+0000","last_became_active":"2024-04-25T13:24:11.883401+0000","last_became_peered":"2024-04-25T13:24:11.883401+0000","last_unstale":"2024-04-25T13:25:07.534372+0000","last_undegraded":"2024-04-25T13:25:07.534372+0000","last_fullsized":"2024-04-25T13:25:07.534372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:04.291865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528716+0000","last_change":"2024-04-25T13:24:10.858303+0000","last_active":"2024-04-25T13:25:07.528716+0000","last_peered":"2024-04-25T13:25:07.528716+0000","last_clean":"2024-04-25T13:25:07.528716+0000","last_became_active":"2024-04-25T13:24:10.857956+0000","last_became_peered":"2024-04-25T13:24:10.857956+0000","last_unstale":"2024-04-25T13:25:07.528716+0000","last_undegraded":"2024-04-25T13:25:07.528716+0000","last_fullsized":"2024-04-25T13:25:07.528716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.753759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535087+0000","last_change":"2024-04-25T13:24:11.882367+0000","last_active":"2024-04-25T13:25:07.535087+0000","last_peered":"2024-04-25T13:25:07.535087+0000","last_clean":"2024-04-25T13:25:07.535087+0000","last_became_active":"2024-04-25T13:24:11.880495+0000","last_became_peered":"2024-04-25T13:24:11.880495+0000","last_unstale":"2024-04-25T13:25:07.535087+0000","last_undegraded":"2024-04-25T13:25:07.535087+0000","last_fullsized":"2024-04-25T13:25:07.535087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.031326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530921+0000","last_change":"2024-04-25T13:24:10.854984+0000","last_active":"2024-04-25T13:25:07.530921+0000","last_peered":"2024-04-25T13:25:07.530921+0000","last_clean":"2024-04-25T13:25:07.530921+0000","last_became_active":"2024-04-25T13:24:10.854691+0000","last_became_peered":"2024-04-25T13:24:10.854691+0000","last_unstale":"2024-04-25T13:25:07.530921+0000","last_undegraded":"2024-04-25T13:25:07.530921+0000","last_fullsized":"2024-04-25T13:25:07.530921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:55.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.534601+0000","last_change":"2024-04-25T13:24:11.881329+0000","last_active":"2024-04-25T13:25:07.534601+0000","last_peered":"2024-04-25T13:25:07.534601+0000","last_clean":"2024-04-25T13:25:07.534601+0000","last_became_active":"2024-04-25T13:24:11.880958+0000","last_became_peered":"2024-04-25T13:24:11.880958+0000","last_unstale":"2024-04-25T13:25:07.534601+0000","last_undegraded":"2024-04-25T13:25:07.534601+0000","last_fullsized":"2024-04-25T13:25:07.534601+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:54.720129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561953+0000","last_change":"2024-04-25T13:24:10.861501+0000","last_active":"2024-04-25T13:25:07.561953+0000","last_peered":"2024-04-25T13:25:07.561953+0000","last_clean":"2024-04-25T13:25:07.561953+0000","last_became_active":"2024-04-25T13:24:10.861155+0000","last_became_peered":"2024-04-25T13:24:10.861155+0000","last_unstale":"2024-04-25T13:25:07.561953+0000","last_undegraded":"2024-04-25T13:25:07.561953+0000","last_fullsized":"2024-04-25T13:25:07.561953+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:09.830297+0000","last_clean_scrub_stamp":"2024-04-25T13:24:09.830297+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:17.306214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.535372+0000","last_change":"2024-04-25T13:24:11.879779+0000","last_active":"2024-04-25T13:25:07.535372+0000","last_peered":"2024-04-25T13:25:07.535372+0000","last_clean":"2024-04-25T13:25:07.535372+0000","last_became_active":"2024-04-25T13:24:11.879285+0000","last_became_peered":"2024-04-25T13:24:11.879285+0000","last_unstale":"2024-04-25T13:25:07.535372+0000","last_undegraded":"2024-04-25T13:25:07.535372+0000","last_fullsized":"2024-04-25T13:25:07.535372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:39.799094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531172+0000","last_change":"2024-04-25T13:24:11.871059+0000","last_active":"2024-04-25T13:25:07.531172+0000","last_peered":"2024-04-25T13:25:07.531172+0000","last_clean":"2024-04-25T13:25:07.531172+0000","last_became_active":"2024-04-25T13:24:11.869530+0000","last_became_peered":"2024-04-25T13:24:11.869530+0000","last_unstale":"2024-04-25T13:25:07.531172+0000","last_undegraded":"2024-04-25T13:25:07.531172+0000","last_fullsized":"2024-04-25T13:25:07.531172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:03.095146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528330+0000","last_change":"2024-04-25T13:24:11.876162+0000","last_active":"2024-04-25T13:25:07.528330+0000","last_peered":"2024-04-25T13:25:07.528330+0000","last_clean":"2024-04-25T13:25:07.528330+0000","last_became_active":"2024-04-25T13:24:11.875747+0000","last_became_peered":"2024-04-25T13:24:11.875747+0000","last_unstale":"2024-04-25T13:25:07.528330+0000","last_undegraded":"2024-04-25T13:25:07.528330+0000","last_fullsized":"2024-04-25T13:25:07.528330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:07.957698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528244+0000","last_change":"2024-04-25T13:24:11.887477+0000","last_active":"2024-04-25T13:25:07.528244+0000","last_peered":"2024-04-25T13:25:07.528244+0000","last_clean":"2024-04-25T13:25:07.528244+0000","last_became_active":"2024-04-25T13:24:11.886951+0000","last_became_peered":"2024-04-25T13:24:11.886951+0000","last_unstale":"2024-04-25T13:25:07.528244+0000","last_undegraded":"2024-04-25T13:25:07.528244+0000","last_fullsized":"2024-04-25T13:25:07.528244+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:28.865817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.563315+0000","last_change":"2024-04-25T13:25:07.563315+0000","last_active":"2024-04-25T13:25:07.563315+0000","last_peered":"2024-04-25T13:25:07.563315+0000","last_clean":"2024-04-25T13:25:07.563315+0000","last_became_active":"2024-04-25T13:25:07.562898+0000","last_became_peered":"2024-04-25T13:25:07.562898+0000","last_unstale":"2024-04-25T13:25:07.563315+0000","last_undegraded":"2024-04-25T13:25:07.563315+0000","last_fullsized":"2024-04-25T13:25:07.563315+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:55.921923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.561859+0000","last_change":"2024-04-25T13:24:11.876929+0000","last_active":"2024-04-25T13:25:07.561859+0000","last_peered":"2024-04-25T13:25:07.561859+0000","last_clean":"2024-04-25T13:25:07.561859+0000","last_became_active":"2024-04-25T13:24:11.876578+0000","last_became_peered":"2024-04-25T13:24:11.876578+0000","last_unstale":"2024-04-25T13:25:07.561859+0000","last_undegraded":"2024-04-25T13:25:07.561859+0000","last_fullsized":"2024-04-25T13:25:07.561859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:24.621282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537307+0000","last_change":"2024-04-25T13:25:07.537307+0000","last_active":"2024-04-25T13:25:07.537307+0000","last_peered":"2024-04-25T13:25:07.537307+0000","last_clean":"2024-04-25T13:25:07.537307+0000","last_became_active":"2024-04-25T13:25:07.536757+0000","last_became_peered":"2024-04-25T13:25:07.536757+0000","last_unstale":"2024-04-25T13:25:07.537307+0000","last_undegraded":"2024-04-25T13:25:07.537307+0000","last_fullsized":"2024-04-25T13:25:07.537307+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:26.143315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.528815+0000","last_change":"2024-04-25T13:24:11.874008+0000","last_active":"2024-04-25T13:25:07.528815+0000","last_peered":"2024-04-25T13:25:07.528815+0000","last_clean":"2024-04-25T13:25:07.528815+0000","last_became_active":"2024-04-25T13:24:11.873576+0000","last_became_peered":"2024-04-25T13:24:11.873576+0000","last_unstale":"2024-04-25T13:25:07.528815+0000","last_undegraded":"2024-04-25T13:25:07.528815+0000","last_fullsized":"2024-04-25T13:25:07.528815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:14.395329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.537492+0000","last_change":"2024-04-25T13:25:07.537492+0000","last_active":"2024-04-25T13:25:07.537492+0000","last_peered":"2024-04-25T13:25:07.537492+0000","last_clean":"2024-04-25T13:25:07.537492+0000","last_became_active":"2024-04-25T13:25:07.537115+0000","last_became_peered":"2024-04-25T13:25:07.537115+0000","last_unstale":"2024-04-25T13:25:07.537492+0000","last_undegraded":"2024-04-25T13:25:07.537492+0000","last_fullsized":"2024-04-25T13:25:07.537492+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:39.084520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530375+0000","last_change":"2024-04-25T13:24:11.885922+0000","last_active":"2024-04-25T13:25:07.530375+0000","last_peered":"2024-04-25T13:25:07.530375+0000","last_clean":"2024-04-25T13:25:07.530375+0000","last_became_active":"2024-04-25T13:24:11.885611+0000","last_became_peered":"2024-04-25T13:24:11.885611+0000","last_unstale":"2024-04-25T13:25:07.530375+0000","last_undegraded":"2024-04-25T13:25:07.530375+0000","last_fullsized":"2024-04-25T13:25:07.530375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:24.857175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.532015+0000","last_change":"2024-04-25T13:24:11.877912+0000","last_active":"2024-04-25T13:25:07.532015+0000","last_peered":"2024-04-25T13:25:07.532015+0000","last_clean":"2024-04-25T13:25:07.532015+0000","last_became_active":"2024-04-25T13:24:11.877139+0000","last_became_peered":"2024-04-25T13:24:11.877139+0000","last_unstale":"2024-04-25T13:25:07.532015+0000","last_undegraded":"2024-04-25T13:25:07.532015+0000","last_fullsized":"2024-04-25T13:25:07.532015+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:45.618556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.530656+0000","last_change":"2024-04-25T13:24:11.889267+0000","last_active":"2024-04-25T13:25:07.530656+0000","last_peered":"2024-04-25T13:25:07.530656+0000","last_clean":"2024-04-25T13:25:07.530656+0000","last_became_active":"2024-04-25T13:24:11.888875+0000","last_became_peered":"2024-04-25T13:24:11.888875+0000","last_unstale":"2024-04-25T13:25:07.530656+0000","last_undegraded":"2024-04-25T13:25:07.530656+0000","last_fullsized":"2024-04-25T13:25:07.530656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:41.871075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.562081+0000","last_change":"2024-04-25T13:24:11.890664+0000","last_active":"2024-04-25T13:25:07.562081+0000","last_peered":"2024-04-25T13:25:07.562081+0000","last_clean":"2024-04-25T13:25:07.562081+0000","last_became_active":"2024-04-25T13:24:11.890124+0000","last_became_peered":"2024-04-25T13:24:11.890124+0000","last_unstale":"2024-04-25T13:25:07.562081+0000","last_undegraded":"2024-04-25T13:25:07.562081+0000","last_fullsized":"2024-04-25T13:25:07.562081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:55.378471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T13:25:07.531595+0000","last_change":"2024-04-25T13:24:11.876201+0000","last_active":"2024-04-25T13:25:07.531595+0000","last_peered":"2024-04-25T13:25:07.531595+0000","last_clean":"2024-04-25T13:25:07.531595+0000","last_became_active":"2024-04-25T13:24:11.875783+0000","last_became_peered":"2024-04-25T13:24:11.875783+0000","last_unstale":"2024-04-25T13:25:07.531595+0000","last_undegraded":"2024-04-25T13:25:07.531595+0000","last_fullsized":"2024-04-25T13:25:07.531595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:24:10.836677+0000","last_clean_scrub_stamp":"2024-04-25T13:24:10.836677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:46.264576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61330,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":76,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":60532,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":183990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":473642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1667072,"data_stored":1124853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":474224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":63,"seq":270582939693,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":474246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234034,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1064709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":473759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":534174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":473732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":473676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":473980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":534196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1597440,"data_stored":1064309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T13:25:31.719 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T13:25:31.720 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:25:31.720 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:25:31.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T13:25:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:32.098 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:31 smithi037 ceph-mon[29204]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:32.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:31 smithi069 ceph-mon[32534]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:32.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:31 smithi044 ceph-mon[32590]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T13:25:33.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:32 smithi037 ceph-mon[29204]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:32 smithi069 ceph-mon[32534]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:33.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:32 smithi044 ceph-mon[32590]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:33.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 -- 172.21.15.37:0/484320111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140101950 msgr2=0x7f314010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:33.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 --2- 172.21.15.37:0/484320111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140101950 0x7f314010f920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f313800b0d0 tx=0x7f313800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 -- 172.21.15.37:0/484320111 shutdown_connections 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 --2- 172.21.15.37:0/484320111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140107a60 0x7f3140107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 --2- 172.21.15.37:0/484320111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140101950 0x7f314010f920 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 --2- 172.21.15.37:0/484320111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 -- 172.21.15.37:0/484320111 >> 172.21.15.37:0/484320111 conn(0x7f3140077f40 msgr2=0x7f31400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 -- 172.21.15.37:0/484320111 shutdown_connections 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.606+0000 7f3146778700 1 -- 172.21.15.37:0/484320111 wait complete. 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.607+0000 7f3146778700 1 Processor -- start 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.607+0000 7f3146778700 1 -- start start 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140101950 0x7f31401996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:33.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112d70 con 0x7f3140107a60 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112bf0 con 0x7f3140101950 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112ef0 con 0x7f3140100f90 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51438/0 (socket says 172.21.15.37:51438) 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 -- 172.21.15.37:0/1540264095 learned_addr learned my addr 172.21.15.37:0/1540264095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f313f7fe700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140101950 0x7f31401996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f313ffff700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 -- 172.21.15.37:0/1540264095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 msgr2=0x7f3140199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 -- 172.21.15.37:0/1540264095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140101950 msgr2=0x7f31401996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140101950 0x7f31401996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 -- 172.21.15.37:0/1540264095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314019e190 con 0x7f3140107a60 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f313ffff700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:33.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3144d15700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3138007a20 tx=0x7f3138007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:33.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313801f070 con 0x7f3140107a60 2024-04-25T13:25:33.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3138003c00 con 0x7f3140107a60 2024-04-25T13:25:33.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.608+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314019e420 con 0x7f3140107a60 2024-04-25T13:25:33.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.609+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313801b6a0 con 0x7f3140107a60 2024-04-25T13:25:33.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.609+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a5d00 con 0x7f3140107a60 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.610+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3140069730 con 0x7f3140107a60 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.611+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3138022070 con 0x7f3140107a60 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.611+0000 7f313d7fa700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 0x7f31280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.611+0000 7f313ffff700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 0x7f31280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.612+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f313800f020 con 0x7f3140107a60 2024-04-25T13:25:33.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.612+0000 7f313ffff700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 0x7f31280a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3130005e10 tx=0x7f3130005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:33.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.615+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3138029020 con 0x7f3140107a60 2024-04-25T13:25:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.873+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f314019a4e0 con 0x7f3140107a60 2024-04-25T13:25:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.874+0000 7f313d7fa700 1 -- 172.21.15.37:0/1540264095 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f313809e440 con 0x7f3140107a60 2024-04-25T13:25:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:33.874 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 msgr2=0x7f31280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 0x7f31280a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3130005e10 tx=0x7f3130005d60 comp rx=0 tx=0).stop 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 msgr2=0x7f314019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3138007a20 tx=0x7f3138007a50 comp rx=0 tx=0).stop 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 shutdown_connections 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f31280a3d50 0x7f31280a6210 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3140107a60 0x7f314019da50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3140100f90 0x7f3140199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 --2- 172.21.15.37:0/1540264095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3140101950 0x7f31401996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 >> 172.21.15.37:0/1540264095 conn(0x7f3140077f40 msgr2=0x7f31400fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 shutdown_connections 2024-04-25T13:25:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:33.876+0000 7f3146778700 1 -- 172.21.15.37:0/1540264095 wait complete. 2024-04-25T13:25:34.521 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:25:34.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.629+0000 7fefd4643640 1 -- 172.21.15.37:0/86859699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc102040 msgr2=0x7fefcc1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.629+0000 7fefd4643640 1 --2- 172.21.15.37:0/86859699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc102040 0x7fefcc1024c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009900 tx=0x7fefbc02a410 comp rx=0 tx=0).stop 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- 172.21.15.37:0/86859699 shutdown_connections 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- 172.21.15.37:0/86859699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 0x7fefcc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- 172.21.15.37:0/86859699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc102040 0x7fefcc1024c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- 172.21.15.37:0/86859699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc100e60 0x7fefcc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- 172.21.15.37:0/86859699 >> 172.21.15.37:0/86859699 conn(0x7fefcc0fc420 msgr2=0x7fefcc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- 172.21.15.37:0/86859699 shutdown_connections 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- 172.21.15.37:0/86859699 wait complete. 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 Processor -- start 2024-04-25T13:25:34.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- start start 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc102040 0x7fefcc1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 0x7fefcc1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc10c9f0 con 0x7fefcc100e60 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc10c870 con 0x7fefcc102040 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd4643640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc10cb70 con 0x7fefcc102a00 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd23df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd23df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51444/0 (socket says 172.21.15.37:51444) 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.630+0000 7fefd23df640 1 -- 172.21.15.37:0/2299164702 learned_addr learned my addr 172.21.15.37:0/2299164702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd2be0640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 0x7fefcc1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd1bde640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc102040 0x7fefcc1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 -- 172.21.15.37:0/2299164702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 msgr2=0x7fefcc1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 0x7fefcc1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 -- 172.21.15.37:0/2299164702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc102040 msgr2=0x7fefcc1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc102040 0x7fefcc1034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 -- 172.21.15.37:0/2299164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc104280 con 0x7fefcc100e60 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd23df640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d950 tx=0x7fefc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017020 con 0x7fefcc100e60 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc1a22f0 con 0x7fefcc100e60 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc8009c30 con 0x7fefcc100e60 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc800f6b0 con 0x7fefcc100e60 2024-04-25T13:25:34.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.631+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc1a2840 con 0x7fefcc100e60 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.632+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc1a2b60 con 0x7fefcc100e60 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.633+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fefc801a020 con 0x7fefcc100e60 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.634+0000 7fefc37fe640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 0x7fefb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.634+0000 7fefd1bde640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 0x7fefb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.634+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fefc80d03c0 con 0x7fefcc100e60 2024-04-25T13:25:34.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.634+0000 7fefd1bde640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 0x7fefb80a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fefbc02a930 tx=0x7fefbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:34.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.638+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefc8009de0 con 0x7fefcc100e60 2024-04-25T13:25:34.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.826+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefcc1a2d30 con 0x7fefcc100e60 2024-04-25T13:25:34.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.828+0000 7fefc37fe640 1 -- 172.21.15.37:0/2299164702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7fefc809cbc0 con 0x7fefcc100e60 2024-04-25T13:25:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:34.829 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 msgr2=0x7fefb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 0x7fefb80a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fefbc02a930 tx=0x7fefbc000bc0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 msgr2=0x7fefcc104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d950 tx=0x7fefc800de10 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 shutdown_connections 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fefcc102a00 0x7fefcc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefcc102040 0x7fefcc1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fefb80a3c40 0x7fefb80a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 --2- 172.21.15.37:0/2299164702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefcc100e60 0x7fefcc104e70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 >> 172.21.15.37:0/2299164702 conn(0x7fefcc0fc420 msgr2=0x7fefcc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:34.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 shutdown_connections 2024-04-25T13:25:34.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.830+0000 7fefd4643640 1 -- 172.21.15.37:0/2299164702 wait complete. 2024-04-25T13:25:34.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:25:34.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:25:34.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.947+0000 7fc23c55d640 1 -- 172.21.15.37:0/1805781126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234102a70 msgr2=0x7fc234104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.947+0000 7fc23c55d640 1 --2- 172.21.15.37:0/1805781126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234102a70 0x7fc234104e60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc230009900 tx=0x7fc23002a410 comp rx=0 tx=0).stop 2024-04-25T13:25:34.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- 172.21.15.37:0/1805781126 shutdown_connections 2024-04-25T13:25:34.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- 172.21.15.37:0/1805781126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234107cd0 0x7fc23410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- 172.21.15.37:0/1805781126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- 172.21.15.37:0/1805781126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234102a70 0x7fc234104e60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- 172.21.15.37:0/1805781126 >> 172.21.15.37:0/1805781126 conn(0x7fc2340fc420 msgr2=0x7fc2340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- 172.21.15.37:0/1805781126 shutdown_connections 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- 172.21.15.37:0/1805781126 wait complete. 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 Processor -- start 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- start start 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234102a70 0x7fc234198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.948+0000 7fc23c55d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc23410dac0 con 0x7fc234107cd0 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23c55d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc23410d940 con 0x7fc234102a70 2024-04-25T13:25:34.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23c55d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc23410dc40 con 0x7fc2341053a0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51466/0 (socket says 172.21.15.37:51466) 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 -- 172.21.15.37:0/554316916 learned_addr learned my addr 172.21.15.37:0/554316916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23a2f9640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234102a70 0x7fc234198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc239af8640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 -- 172.21.15.37:0/554316916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 msgr2=0x7fc234198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 -- 172.21.15.37:0/554316916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234102a70 msgr2=0x7fc234198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234102a70 0x7fc234198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 -- 172.21.15.37:0/554316916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23419fe70 con 0x7fc234107cd0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc239af8640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23aafa640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00eb10 tx=0x7fc22c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:34.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004b20 con 0x7fc234107cd0 2024-04-25T13:25:34.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22c004cc0 con 0x7fc234107cd0 2024-04-25T13:25:34.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2341a0160 con 0x7fc234107cd0 2024-04-25T13:25:34.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c010670 con 0x7fc234107cd0 2024-04-25T13:25:34.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.949+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2341a0660 con 0x7fc234107cd0 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.950+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2340fdf10 con 0x7fc234107cd0 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.951+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc22c005020 con 0x7fc234107cd0 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.952+0000 7fc22b7fe640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 0x7fc2200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.952+0000 7fc23a2f9640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 0x7fc2200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.952+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fc22c0d0cf0 con 0x7fc234107cd0 2024-04-25T13:25:34.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.953+0000 7fc23a2f9640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 0x7fc2200a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc230000c00 tx=0x7fc2300314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:34.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:34.957+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc22c09d4f0 con 0x7fc234107cd0 2024-04-25T13:25:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:34 smithi037 ceph-mon[29204]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:34 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1540264095' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:25:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:34 smithi069 ceph-mon[32534]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:34 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1540264095' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:25:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:34 smithi044 ceph-mon[32590]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:34 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1540264095' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T13:25:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.144+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc234062b50 con 0x7fc234107cd0 2024-04-25T13:25:35.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.145+0000 7fc22b7fe640 1 -- 172.21.15.37:0/554316916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fc22c09cdf0 con 0x7fc234107cd0 2024-04-25T13:25:35.145 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:25:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 msgr2=0x7fc2200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 0x7fc2200a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc230000c00 tx=0x7fc2300314a0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 msgr2=0x7fc23419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00eb10 tx=0x7fc22c00c2d0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 shutdown_connections 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fc2200a3c40 0x7fc2200a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc234107cd0 0x7fc23419f800 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2341053a0 0x7fc234198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 --2- 172.21.15.37:0/554316916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc234102a70 0x7fc234198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 >> 172.21.15.37:0/554316916 conn(0x7fc2340fc420 msgr2=0x7fc234104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 shutdown_connections 2024-04-25T13:25:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.147+0000 7fc23c55d640 1 -- 172.21.15.37:0/554316916 wait complete. 2024-04-25T13:25:35.159 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 -- 172.21.15.37:0/3483848058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 msgr2=0x7f7338102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 --2- 172.21.15.37:0/3483848058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338102440 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7320009900 tx=0x7f732002a410 comp rx=0 tx=0).stop 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 -- 172.21.15.37:0/3483848058 shutdown_connections 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 --2- 172.21.15.37:0/3483848058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338102980 0x7f73381092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 --2- 172.21.15.37:0/3483848058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338102440 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 --2- 172.21.15.37:0/3483848058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338100e70 0x7f7338101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.262+0000 7f733db43640 1 -- 172.21.15.37:0/3483848058 >> 172.21.15.37:0/3483848058 conn(0x7f73380fc3c0 msgr2=0x7f73380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- 172.21.15.37:0/3483848058 shutdown_connections 2024-04-25T13:25:35.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- 172.21.15.37:0/3483848058 wait complete. 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 Processor -- start 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- start start 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338100e70 0x7f7338198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338102980 0x7f733819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f733810c9a0 con 0x7f7338101fc0 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f733810c820 con 0x7f7338102980 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.263+0000 7f733db43640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f733810cb20 con 0x7f7338100e70 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51480/0 (socket says 172.21.15.37:51480) 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 -- 172.21.15.37:0/1643160633 learned_addr learned my addr 172.21.15.37:0/1643160633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f7337fff640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338102980 0x7f733819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f73377fe640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338100e70 0x7f7338198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 -- 172.21.15.37:0/1643160633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338100e70 msgr2=0x7f7338198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338100e70 0x7f7338198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 -- 172.21.15.37:0/1643160633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338102980 msgr2=0x7f733819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338102980 0x7f733819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 -- 172.21.15.37:0/1643160633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733819fdf0 con 0x7f7338101fc0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f732effd640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7320006010 tx=0x7f73200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732003f030 con 0x7f7338101fc0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73200318f0 con 0x7f7338101fc0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381a0080 con 0x7f7338101fc0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320035070 con 0x7f7338101fc0 2024-04-25T13:25:35.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.264+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a05c0 con 0x7f7338101fc0 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.265+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73381a0820 con 0x7f7338101fc0 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.267+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7320042030 con 0x7f7338101fc0 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.267+0000 7f73357fa640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 0x7f73240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.267+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f73200f0de0 con 0x7f7338101fc0 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.267+0000 7f73377fe640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 0x7f73240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.268+0000 7f73377fe640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 0x7f73240a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7328009bd0 tx=0x7f7328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.271+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73200bcff0 con 0x7f7338101fc0 2024-04-25T13:25:35.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.456+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73381a09f0 con 0x7f7338101fc0 2024-04-25T13:25:35.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.456+0000 7f73357fa640 1 -- 172.21.15.37:0/1643160633 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f73200c12f0 con 0x7f7338101fc0 2024-04-25T13:25:35.457 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 msgr2=0x7f73240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 0x7f73240a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7328009bd0 tx=0x7f7328005f90 comp rx=0 tx=0).stop 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 msgr2=0x7f7338198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7320006010 tx=0x7f73200042f0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 shutdown_connections 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7338102980 0x7f733819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f73240a3cc0 0x7f73240a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7338101fc0 0x7f7338198760 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 --2- 172.21.15.37:0/1643160633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7338100e70 0x7f7338198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.458+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 >> 172.21.15.37:0/1643160633 conn(0x7f73380fc3c0 msgr2=0x7f73380fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.459+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 shutdown_connections 2024-04-25T13:25:35.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.459+0000 7f733db43640 1 -- 172.21.15.37:0/1643160633 wait complete. 2024-04-25T13:25:35.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 -- 172.21.15.37:0/906853921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710100630 msgr2=0x7fd710100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 --2- 172.21.15.37:0/906853921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710100630 0x7fd710100a50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd70000aa10 tx=0x7fd70002b720 comp rx=0 tx=0).stop 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 -- 172.21.15.37:0/906853921 shutdown_connections 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 --2- 172.21.15.37:0/906853921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7101021e0 0x7fd710108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 --2- 172.21.15.37:0/906853921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd710101820 0x7fd710101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 --2- 172.21.15.37:0/906853921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710100630 0x7fd710100a50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 -- 172.21.15.37:0/906853921 >> 172.21.15.37:0/906853921 conn(0x7fd7100fbb80 msgr2=0x7fd7100fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 -- 172.21.15.37:0/906853921 shutdown_connections 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.574+0000 7fd7150bb640 1 -- 172.21.15.37:0/906853921 wait complete. 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 Processor -- start 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 -- start start 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd710100630 0x7fd71019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7101021e0 0x7fd7101a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010b8f0 con 0x7fd710101820 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010b770 con 0x7fd7101021e0 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd7150bb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010ba70 con 0x7fd710100630 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd707fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd707fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51510/0 (socket says 172.21.15.37:51510) 2024-04-25T13:25:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd707fff640 1 -- 172.21.15.37:0/3494819777 learned_addr learned my addr 172.21.15.37:0/3494819777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd70ed9d640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd710100630 0x7fd71019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.575+0000 7fd70f59e640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7101021e0 0x7fd7101a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 -- 172.21.15.37:0/3494819777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd710100630 msgr2=0x7fd71019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd710100630 0x7fd71019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 -- 172.21.15.37:0/3494819777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7101021e0 msgr2=0x7fd7101a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7101021e0 0x7fd7101a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 -- 172.21.15.37:0/3494819777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7101a2020 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd707fff640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800d950 tx=0x7fd6f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8017070 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6f8009c30 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7101a2310 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f800f620 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.576+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a28b0 con 0x7fd710101820 2024-04-25T13:25:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.577+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710107980 con 0x7fd710101820 2024-04-25T13:25:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.578+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6f8010460 con 0x7fd710101820 2024-04-25T13:25:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.579+0000 7fd70cd99640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 0x7fd6fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.579+0000 7fd70ed9d640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 0x7fd6fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.579+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd6f80d0070 con 0x7fd710101820 2024-04-25T13:25:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.579+0000 7fd70ed9d640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 0x7fd6fc0a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd70000a340 tx=0x7fd7000305b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.583+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6f8009de0 con 0x7fd710101820 2024-04-25T13:25:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.798+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd71002d750 con 0x7fd710101820 2024-04-25T13:25:35.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.799+0000 7fd70cd99640 1 -- 172.21.15.37:0/3494819777 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fd6f809c320 con 0x7fd710101820 2024-04-25T13:25:35.799 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:25:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 msgr2=0x7fd6fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 0x7fd6fc0a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd70000a340 tx=0x7fd7000305b0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 msgr2=0x7fd71019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800d950 tx=0x7fd6f800de10 comp rx=0 tx=0).stop 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 shutdown_connections 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7101021e0 0x7fd7101a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6fc0a3cc0 0x7fd6fc0a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd710101820 0x7fd71019a990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 --2- 172.21.15.37:0/3494819777 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd710100630 0x7fd71019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 >> 172.21.15.37:0/3494819777 conn(0x7fd7100fbb80 msgr2=0x7fd710104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 shutdown_connections 2024-04-25T13:25:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.801+0000 7fd7150bb640 1 -- 172.21.15.37:0/3494819777 wait complete. 2024-04-25T13:25:35.814 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:25:35.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:35 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2299164702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:35.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:35 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/554316916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:35.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:35 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1643160633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- 172.21.15.37:0/1895075642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 msgr2=0x7fd178103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1895075642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178103620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002a410 comp rx=0 tx=0).stop 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- 172.21.15.37:0/1895075642 shutdown_connections 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1895075642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178103b60 0x7fd17810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1895075642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178103620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1895075642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178101fb0 0x7fd1781023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- 172.21.15.37:0/1895075642 >> 172.21.15.37:0/1895075642 conn(0x7fd1780fd540 msgr2=0x7fd1780ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:35.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- 172.21.15.37:0/1895075642 shutdown_connections 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- 172.21.15.37:0/1895075642 wait complete. 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 Processor -- start 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.916+0000 7fd17ec31640 1 -- start start 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 0x7fd178075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178103b60 0x7fd178074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810daf0 con 0x7fd1781031a0 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810d970 con 0x7fd178101fb0 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810dc70 con 0x7fd178103b60 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51528/0 (socket says 172.21.15.37:51528) 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 -- 172.21.15.37:0/1553252618 learned_addr learned my addr 172.21.15.37:0/1553252618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17c9cd640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 0x7fd178075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17d1ce640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178103b60 0x7fd178074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 -- 172.21.15.37:0/1553252618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178103b60 msgr2=0x7fd178074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178103b60 0x7fd178074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 -- 172.21.15.37:0/1553252618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 msgr2=0x7fd178075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 0x7fd178075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 -- 172.21.15.37:0/1553252618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd178075110 con 0x7fd1781031a0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17c9cd640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 0x7fd178075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd177fff640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd160005940 tx=0x7fd160004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16003f030 con 0x7fd1781031a0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd160003cb0 con 0x7fd1781031a0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1781a2200 con 0x7fd1781031a0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.917+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160035070 con 0x7fd1781031a0 2024-04-25T13:25:35.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.918+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a27a0 con 0x7fd1781031a0 2024-04-25T13:25:35.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.918+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd178109320 con 0x7fd1781031a0 2024-04-25T13:25:35.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.920+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd160042030 con 0x7fd1781031a0 2024-04-25T13:25:35.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.920+0000 7fd175ffb640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 0x7fd1640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:35.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.921+0000 7fd17c9cd640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 0x7fd1640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:35.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.921+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd1600f0f30 con 0x7fd1781031a0 2024-04-25T13:25:35.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.921+0000 7fd17c9cd640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 0x7fd1640a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd1680058c0 tx=0x7fd168005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:35.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:35.925+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd160003e60 con 0x7fd1781031a0 2024-04-25T13:25:36.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:35 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2299164702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:36.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:35 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/554316916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:36.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:35 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1643160633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:36.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:35 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2299164702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:36.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:35 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/554316916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:36.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:35 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1643160633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:36.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.152+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd17802d5c0 con 0x7fd1781031a0 2024-04-25T13:25:36.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.153+0000 7fd175ffb640 1 -- 172.21.15.37:0/1553252618 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7fd1600bd790 con 0x7fd1781031a0 2024-04-25T13:25:36.153 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:36.153 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:25:36.153 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:25:36.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.154+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 msgr2=0x7fd1640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.154+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 0x7fd1640a6180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd1680058c0 tx=0x7fd168005850 comp rx=0 tx=0).stop 2024-04-25T13:25:36.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.154+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 msgr2=0x7fd178074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.154+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd160005940 tx=0x7fd160004430 comp rx=0 tx=0).stop 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 shutdown_connections 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd178103b60 0x7fd178074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd1640a3cc0 0x7fd1640a6180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1781031a0 0x7fd178074340 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 --2- 172.21.15.37:0/1553252618 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd178101fb0 0x7fd178075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 >> 172.21.15.37:0/1553252618 conn(0x7fd1780fd540 msgr2=0x7fd1781064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 shutdown_connections 2024-04-25T13:25:36.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.155+0000 7fd17ec31640 1 -- 172.21.15.37:0/1553252618 wait complete. 2024-04-25T13:25:36.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 -- 172.21.15.37:0/3452639370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0103b60 msgr2=0x7f07e010a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 --2- 172.21.15.37:0/3452639370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0103b60 0x7f07e010a3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009ca0 tx=0x7f07d802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 -- 172.21.15.37:0/3452639370 shutdown_connections 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 --2- 172.21.15.37:0/3452639370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0103b60 0x7f07e010a3f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 --2- 172.21.15.37:0/3452639370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e01031a0 0x7f07e0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 --2- 172.21.15.37:0/3452639370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07e0101fb0 0x7f07e01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 -- 172.21.15.37:0/3452639370 >> 172.21.15.37:0/3452639370 conn(0x7f07e00fd540 msgr2=0x7f07e00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 -- 172.21.15.37:0/3452639370 shutdown_connections 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.268+0000 7f07e532d640 1 -- 172.21.15.37:0/3452639370 wait complete. 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 Processor -- start 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 -- start start 2024-04-25T13:25:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e01031a0 0x7f07e0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07e0103b60 0x7f07e0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010daf0 con 0x7f07e0101fb0 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010d970 con 0x7f07e01031a0 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07e532d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07e010dc70 con 0x7f07e0103b60 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51542/0 (socket says 172.21.15.37:51542) 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 -- 172.21.15.37:0/2273056909 learned_addr learned my addr 172.21.15.37:0/2273056909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 -- 172.21.15.37:0/2273056909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07e0103b60 msgr2=0x7f07e0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07e0103b60 0x7f07e0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 -- 172.21.15.37:0/2273056909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e01031a0 msgr2=0x7f07e0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e01031a0 0x7f07e0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 -- 172.21.15.37:0/2273056909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e0075110 con 0x7f07e0101fb0 2024-04-25T13:25:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.269+0000 7f07deffd640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c970 tx=0x7f07d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004220 con 0x7f07e0101fb0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d00043c0 con 0x7f07e0101fb0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01a2250 con 0x7f07e0101fb0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004a00 con 0x7f07e0101fb0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01a27a0 con 0x7f07e0101fb0 2024-04-25T13:25:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.270+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0109270 con 0x7f07e0101fb0 2024-04-25T13:25:36.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.272+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07d00079b0 con 0x7f07e0101fb0 2024-04-25T13:25:36.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.272+0000 7f07dcff9640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 0x7f07cc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.272+0000 7f07d7fff640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 0x7f07cc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.272+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f07d00d07e0 con 0x7f07e0101fb0 2024-04-25T13:25:36.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.273+0000 7f07d7fff640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 0x7f07cc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07e0075a70 tx=0x7f07c8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.276+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d0014030 con 0x7f07e0101fb0 2024-04-25T13:25:36.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.462+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e002d800 con 0x7f07e0101fb0 2024-04-25T13:25:36.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.464+0000 7f07dcff9640 1 -- 172.21.15.37:0/2273056909 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7f07d009d0c0 con 0x7f07e0101fb0 2024-04-25T13:25:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:36.465 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 msgr2=0x7f07cc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 0x7f07cc0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07e0075a70 tx=0x7f07c8005c70 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 msgr2=0x7f07e0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c970 tx=0x7f07d000ce30 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 shutdown_connections 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07e0103b60 0x7f07e0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07e01031a0 0x7f07e0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07cc0a3cc0 0x7f07cc0a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 --2- 172.21.15.37:0/2273056909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07e0101fb0 0x7f07e0075d00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 >> 172.21.15.37:0/2273056909 conn(0x7f07e00fd540 msgr2=0x7f07e0106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 shutdown_connections 2024-04-25T13:25:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.466+0000 7f07e532d640 1 -- 172.21.15.37:0/2273056909 wait complete. 2024-04-25T13:25:36.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:25:36.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:25:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 -- 172.21.15.37:0/422998900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 msgr2=0x7ffb10102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 --2- 172.21.15.37:0/422998900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10102460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009900 tx=0x7ffb0402a410 comp rx=0 tx=0).stop 2024-04-25T13:25:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 -- 172.21.15.37:0/422998900 shutdown_connections 2024-04-25T13:25:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 --2- 172.21.15.37:0/422998900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb101029a0 0x7ffb101092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 --2- 172.21.15.37:0/422998900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10102460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 --2- 172.21.15.37:0/422998900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb10100e90 0x7ffb101012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.582+0000 7ffb168fd640 1 -- 172.21.15.37:0/422998900 >> 172.21.15.37:0/422998900 conn(0x7ffb100fc420 msgr2=0x7ffb100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- 172.21.15.37:0/422998900 shutdown_connections 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- 172.21.15.37:0/422998900 wait complete. 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 Processor -- start 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- start start 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb10100e90 0x7ffb10104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb101029a0 0x7ffb10103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010c9e0 con 0x7ffb10101fe0 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010c860 con 0x7ffb101029a0 2024-04-25T13:25:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.583+0000 7ffb168fd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1010cb60 con 0x7ffb10100e90 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51570/0 (socket says 172.21.15.37:51570) 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 -- 172.21.15.37:0/2791246538 learned_addr learned my addr 172.21.15.37:0/2791246538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb14e9a640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb101029a0 0x7ffb10103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb10100e90 0x7ffb10104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 -- 172.21.15.37:0/2791246538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb10100e90 msgr2=0x7ffb10104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb10100e90 0x7ffb10104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 -- 172.21.15.37:0/2791246538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb101029a0 msgr2=0x7ffb10103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb101029a0 0x7ffb10103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 -- 172.21.15.37:0/2791246538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb10104220 con 0x7ffb10101fe0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0f7fe640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006010 tx=0x7ffb040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0403f030 con 0x7ffb10101fe0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb101a22a0 con 0x7ffb10101fe0 2024-04-25T13:25:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb04003cb0 con 0x7ffb10101fe0 2024-04-25T13:25:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04035070 con 0x7ffb10101fe0 2024-04-25T13:25:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.584+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a27f0 con 0x7ffb10101fe0 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.585+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb101a2a50 con 0x7ffb10101fe0 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.586+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb04042030 con 0x7ffb10101fe0 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.587+0000 7ffb0d7fa640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 0x7ffaf80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.587+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 0x7ffaf80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.587+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7ffb040f0b40 con 0x7ffb10101fe0 2024-04-25T13:25:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.587+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 0x7ffaf80a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000bbe0 tx=0x7ffb0000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.592+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb040bcd50 con 0x7ffb10101fe0 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:36 smithi037 ceph-mon[29204]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:36 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3494819777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:36 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1553252618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:36 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2273056909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:36.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.778+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb101a2c20 con 0x7ffb10101fe0 2024-04-25T13:25:36.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.787+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffb040b93c0 con 0x7ffb10101fe0 2024-04-25T13:25:36.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.788+0000 7ffb0d7fa640 1 -- 172.21.15.37:0/2791246538 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ffb040adef0 con 0x7ffb10101fe0 2024-04-25T13:25:36.789 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.790+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 msgr2=0x7ffaf80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.790+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 0x7ffaf80a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000bbe0 tx=0x7ffb0000b390 comp rx=0 tx=0).stop 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.790+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 msgr2=0x7ffb10103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.790+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb04006010 tx=0x7ffb040042f0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 shutdown_connections 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffb101029a0 0x7ffb10103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7ffaf80a3c40 0x7ffaf80a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101fe0 0x7ffb10103450 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 --2- 172.21.15.37:0/2791246538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb10100e90 0x7ffb10104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 >> 172.21.15.37:0/2791246538 conn(0x7ffb100fc420 msgr2=0x7ffb100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 shutdown_connections 2024-04-25T13:25:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.791+0000 7ffb168fd640 1 -- 172.21.15.37:0/2791246538 wait complete. 2024-04-25T13:25:36.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:25:36.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/3199481499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103b40 msgr2=0x7f9d6810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/3199481499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103b40 0x7f9d6810a3d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009ca0 tx=0x7f9d6002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/3199481499 shutdown_connections 2024-04-25T13:25:36.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/3199481499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103b40 0x7f9d6810a3d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/3199481499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68103180 0x7f9d68103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/3199481499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68101f90 0x7f9d681023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/3199481499 >> 172.21.15.37:0/3199481499 conn(0x7f9d680fd4e0 msgr2=0x7f9d680ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/3199481499 shutdown_connections 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.906+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/3199481499 wait complete. 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 Processor -- start 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 -- start start 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68101f90 0x7f9d68075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68103b40 0x7f9d68074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6810dad0 con 0x7f9d68103180 2024-04-25T13:25:36.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6810d950 con 0x7f9d68103b40 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d6dcd0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6810dc50 con 0x7f9d68101f90 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51586/0 (socket says 172.21.15.37:51586) 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d66ffd640 1 -- 172.21.15.37:0/2905308420 learned_addr learned my addr 172.21.15.37:0/2905308420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d67fff640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68103b40 0x7f9d68074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d677fe640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68101f90 0x7f9d68075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d66ffd640 1 -- 172.21.15.37:0/2905308420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68101f90 msgr2=0x7f9d68075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.907+0000 7f9d66ffd640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68101f90 0x7f9d68075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d66ffd640 1 -- 172.21.15.37:0/2905308420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68103b40 msgr2=0x7f9d68074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d66ffd640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68103b40 0x7f9d68074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d66ffd640 1 -- 172.21.15.37:0/2905308420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d68075110 con 0x7f9d68103180 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d66ffd640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c920 tx=0x7f9d5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c007c70 con 0x7f9d68103180 2024-04-25T13:25:36.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5c007e10 con 0x7f9d68103180 2024-04-25T13:25:36.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d681a2170 con 0x7f9d68103180 2024-04-25T13:25:36.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c018820 con 0x7f9d68103180 2024-04-25T13:25:36.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.908+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a26c0 con 0x7f9d68103180 2024-04-25T13:25:36.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.909+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d681091e0 con 0x7f9d68103180 2024-04-25T13:25:36.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.910+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9d5c00f450 con 0x7f9d68103180 2024-04-25T13:25:36.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.911+0000 7f9d64ff9640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:36.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.911+0000 7f9d677fe640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:36.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.911+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f9d5c0d0590 con 0x7f9d68103180 2024-04-25T13:25:36.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.911+0000 7f9d677fe640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 0x7f9d500a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d68102fe0 tx=0x7f9d58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:36.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:36.916+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d5c014030 con 0x7f9d68103180 2024-04-25T13:25:37.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.099+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d68062b50 con 0x7f9d68103180 2024-04-25T13:25:37.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.100+0000 7f9d64ff9640 1 -- 172.21.15.37:0/2905308420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f9d5c09cef0 con 0x7f9d68103180 2024-04-25T13:25:37.101 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:25:37.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 msgr2=0x7f9d500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 0x7f9d500a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d68102fe0 tx=0x7f9d58005f90 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 msgr2=0x7f9d68074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c920 tx=0x7f9d5c00cde0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 shutdown_connections 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d68103b40 0x7f9d68074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9d500a3bb0 0x7f9d500a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68103180 0x7f9d68074340 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 --2- 172.21.15.37:0/2905308420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d68101f90 0x7f9d68075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 >> 172.21.15.37:0/2905308420 conn(0x7f9d680fd4e0 msgr2=0x7f9d681063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 shutdown_connections 2024-04-25T13:25:37.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.102+0000 7f9d6dcd0640 1 -- 172.21.15.37:0/2905308420 wait complete. 2024-04-25T13:25:37.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:25:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:36 smithi069 ceph-mon[32534]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:36 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3494819777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:36 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1553252618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:36 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2273056909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:36 smithi044 ceph-mon[32590]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:36 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3494819777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:37.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:36 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1553252618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:37.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:36 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2273056909' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 -- 172.21.15.37:0/2462439199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248109e60 msgr2=0x7f424810c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 --2- 172.21.15.37:0/2462439199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248109e60 0x7f424810c2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4240009ca0 tx=0x7f424002a950 comp rx=0 tx=0).stop 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 -- 172.21.15.37:0/2462439199 shutdown_connections 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 --2- 172.21.15.37:0/2462439199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248109e60 0x7f424810c2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 --2- 172.21.15.37:0/2462439199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248074930 0x7f4248074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 --2- 172.21.15.37:0/2462439199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248075f10 0x7f4248074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 -- 172.21.15.37:0/2462439199 >> 172.21.15.37:0/2462439199 conn(0x7f42480fd520 msgr2=0x7f42480ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 -- 172.21.15.37:0/2462439199 shutdown_connections 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.217+0000 7f424dce1640 1 -- 172.21.15.37:0/2462439199 wait complete. 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 Processor -- start 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 -- start start 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248075f10 0x7f424819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 0x7f42481a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f424810fd00 con 0x7f4248074930 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f424810fb80 con 0x7f4248075f10 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f424dce1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f424810fe80 con 0x7f4248109e60 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51594/0 (socket says 172.21.15.37:51594) 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 -- 172.21.15.37:0/73907794 learned_addr learned my addr 172.21.15.37:0/73907794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f4247fff640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 0x7f42481a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 -- 172.21.15.37:0/73907794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 msgr2=0x7f42481a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 0x7f42481a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 -- 172.21.15.37:0/73907794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248075f10 msgr2=0x7f424819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248075f10 0x7f424819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f42477fe640 1 -- 172.21.15.37:0/73907794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42481a1ef0 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.218+0000 7f4247fff640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 0x7f42481a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f42477fe640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f423800d950 tx=0x7f423800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238017070 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4238009c30 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42481a21e0 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423800f620 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.219+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a2730 con 0x7f4248074930 2024-04-25T13:25:37.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.220+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4248108c20 con 0x7f4248074930 2024-04-25T13:25:37.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.221+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4238010460 con 0x7f4248074930 2024-04-25T13:25:37.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.221+0000 7f4244ff9640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.222+0000 7f4246ffd640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.222+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f42380d0100 con 0x7f4248074930 2024-04-25T13:25:37.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.222+0000 7f4246ffd640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 0x7f42300a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f424819b7e0 tx=0x7f423c005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.226+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f423809cac0 con 0x7f4248074930 2024-04-25T13:25:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.414+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4248062b80 con 0x7f4248074930 2024-04-25T13:25:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.414+0000 7f4244ff9640 1 -- 172.21.15.37:0/73907794 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f4238009de0 con 0x7f4248074930 2024-04-25T13:25:37.415 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 msgr2=0x7f42300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 0x7f42300a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f424819b7e0 tx=0x7f423c005c70 comp rx=0 tx=0).stop 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 msgr2=0x7f424819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f423800d950 tx=0x7f423800de10 comp rx=0 tx=0).stop 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 shutdown_connections 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4248109e60 0x7f42481a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4248075f10 0x7f424819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f42300a3bb0 0x7f42300a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 --2- 172.21.15.37:0/73907794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248074930 0x7f424819a2d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.416+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 >> 172.21.15.37:0/73907794 conn(0x7f42480fd520 msgr2=0x7f4248105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.417+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 shutdown_connections 2024-04-25T13:25:37.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.417+0000 7f424dce1640 1 -- 172.21.15.37:0/73907794 wait complete. 2024-04-25T13:25:37.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 -- 172.21.15.37:0/3477725059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 msgr2=0x7f28c81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 --2- 172.21.15.37:0/3477725059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81077d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009900 tx=0x7f28b802a410 comp rx=0 tx=0).stop 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 -- 172.21.15.37:0/3477725059 shutdown_connections 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 --2- 172.21.15.37:0/3477725059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 0x7f28c810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 --2- 172.21.15.37:0/3477725059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81077d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 --2- 172.21.15.37:0/3477725059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 0x7f28c8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 -- 172.21.15.37:0/3477725059 >> 172.21.15.37:0/3477725059 conn(0x7f28c80fc420 msgr2=0x7f28c80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 -- 172.21.15.37:0/3477725059 shutdown_connections 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.529+0000 7f28d06bb640 1 -- 172.21.15.37:0/3477725059 wait complete. 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 Processor -- start 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 -- start start 2024-04-25T13:25:37.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 0x7f28c81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 0x7f28c819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810dab0 con 0x7f28c81053e0 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810d930 con 0x7f28c8107d10 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28d06bb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28c810dc30 con 0x7f28c8102ab0 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28cdc56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28cdc56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51608/0 (socket says 172.21.15.37:51608) 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28cdc56640 1 -- 172.21.15.37:0/648676644 learned_addr learned my addr 172.21.15.37:0/648676644 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28ce457640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 0x7f28c81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.530+0000 7f28cec58640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 0x7f28c819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 -- 172.21.15.37:0/648676644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 msgr2=0x7f28c81981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 0x7f28c81981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 -- 172.21.15.37:0/648676644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 msgr2=0x7f28c819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 0x7f28c819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 -- 172.21.15.37:0/648676644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c819fdd0 con 0x7f28c81053e0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28cdc56640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006010 tx=0x7f28b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b803f030 con 0x7f28c81053e0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8003cb0 con 0x7f28c81053e0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c81a0060 con 0x7f28c81053e0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8035070 con 0x7f28c81053e0 2024-04-25T13:25:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.531+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c81a05a0 con 0x7f28c81053e0 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.532+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c80fdf10 con 0x7f28c81053e0 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.533+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f28b8042030 con 0x7f28c81053e0 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.534+0000 7f28bf7fe640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 0x7f28b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.534+0000 7f28ce457640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 0x7f28b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.534+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f28b80f0a00 con 0x7f28c81053e0 2024-04-25T13:25:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.534+0000 7f28ce457640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 0x7f28b40a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005d00 tx=0x7f28c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.538+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28b80bd3c0 con 0x7f28c81053e0 2024-04-25T13:25:37.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.765+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c8062b50 con 0x7f28c81053e0 2024-04-25T13:25:37.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.766+0000 7f28bf7fe640 1 -- 172.21.15.37:0/648676644 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7f28b80bccc0 con 0x7f28c81053e0 2024-04-25T13:25:37.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:37.766 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:25:37.766 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.767+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 msgr2=0x7f28b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.767+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 0x7f28b40a60f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005d00 tx=0x7f28c400b410 comp rx=0 tx=0).stop 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.767+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 msgr2=0x7f28c81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.767+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006010 tx=0x7f28b80042f0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 shutdown_connections 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28c8107d10 0x7f28c819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f28b40a3c30 0x7f28b40a60f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28c81053e0 0x7f28c81986f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 --2- 172.21.15.37:0/648676644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28c8102ab0 0x7f28c81981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 >> 172.21.15.37:0/648676644 conn(0x7f28c80fc420 msgr2=0x7f28c81034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 shutdown_connections 2024-04-25T13:25:37.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.768+0000 7f28d06bb640 1 -- 172.21.15.37:0/648676644 wait complete. 2024-04-25T13:25:37.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:25:37.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 -- 172.21.15.37:0/4012752893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818101180 msgr2=0x7fb818101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/4012752893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818101180 0x7fb818101600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009900 tx=0x7fb80c02a410 comp rx=0 tx=0).stop 2024-04-25T13:25:37.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 -- 172.21.15.37:0/4012752893 shutdown_connections 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/4012752893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 0x7fb8181093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/4012752893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818101180 0x7fb818101600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/4012752893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818102450 0x7fb818102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 -- 172.21.15.37:0/4012752893 >> 172.21.15.37:0/4012752893 conn(0x7fb8180fbbb0 msgr2=0x7fb8180fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 -- 172.21.15.37:0/4012752893 shutdown_connections 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.881+0000 7fb81cc0b640 1 -- 172.21.15.37:0/4012752893 wait complete. 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 Processor -- start 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 -- start start 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 0x7fb8181982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 0x7fb8181987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810db80 con 0x7fb818102450 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810da00 con 0x7fb818101b40 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81cc0b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb81810dd00 con 0x7fb818101180 2024-04-25T13:25:37.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51618/0 (socket says 172.21.15.37:51618) 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 -- 172.21.15.37:0/676627953 learned_addr learned my addr 172.21.15.37:0/676627953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81659c640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 0x7fb8181982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 -- 172.21.15.37:0/676627953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 msgr2=0x7fb8181982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb815d9b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 0x7fb8181987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 0x7fb8181982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 -- 172.21.15.37:0/676627953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 msgr2=0x7fb8181987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 0x7fb8181987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 -- 172.21.15.37:0/676627953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81819fed0 con 0x7fb818102450 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb81659c640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 0x7fb8181982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb816d9d640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb81000eaa0 tx=0x7fb81000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.882+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81000cdb0 con 0x7fb818102450 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.883+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb810005020 con 0x7fb818102450 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.883+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8100116d0 con 0x7fb818102450 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.883+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8181a01c0 con 0x7fb818102450 2024-04-25T13:25:37.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.883+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8181a0790 con 0x7fb818102450 2024-04-25T13:25:37.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.884+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb818108340 con 0x7fb818102450 2024-04-25T13:25:37.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.885+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb8100051d0 con 0x7fb818102450 2024-04-25T13:25:37.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.885+0000 7fb8077fe640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 0x7fb8000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:37.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.885+0000 7fb81659c640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 0x7fb8000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:37.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.885+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fb8100d0660 con 0x7fb818102450 2024-04-25T13:25:37.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.886+0000 7fb81659c640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 0x7fb8000a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb81806a3e0 tx=0x7fb808008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:37.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:37.890+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb810016030 con 0x7fb818102450 2024-04-25T13:25:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.077+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb818062b50 con 0x7fb818102450 2024-04-25T13:25:38.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.079+0000 7fb8077fe640 1 -- 172.21.15.37:0/676627953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7fb81009d020 con 0x7fb818102450 2024-04-25T13:25:38.079 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:38.079 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:25:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 msgr2=0x7fb8000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 0x7fb8000a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb81806a3e0 tx=0x7fb808008040 comp rx=0 tx=0).stop 2024-04-25T13:25:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 msgr2=0x7fb81819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb81000eaa0 tx=0x7fb81000ef60 comp rx=0 tx=0).stop 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 shutdown_connections 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fb8000a3c30 0x7fb8000a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb818102450 0x7fb81819f860 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb818101b40 0x7fb8181987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 --2- 172.21.15.37:0/676627953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb818101180 0x7fb8181982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 >> 172.21.15.37:0/676627953 conn(0x7fb8180fbbb0 msgr2=0x7fb818105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 shutdown_connections 2024-04-25T13:25:38.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.081+0000 7fb81cc0b640 1 -- 172.21.15.37:0/676627953 wait complete. 2024-04-25T13:25:38.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:37 smithi037 ceph-mon[29204]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:25:38.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:37 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2791246538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:37 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2905308420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.092 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:37 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/73907794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.092 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:37 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/648676644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:38.094 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:25:38.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:37 smithi069 ceph-mon[32534]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:25:38.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:37 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2791246538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:37 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2905308420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:37 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/73907794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:37 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/648676644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:38.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:37 smithi044 ceph-mon[32590]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T13:25:38.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:37 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2791246538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:37 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2905308420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:37 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/73907794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:25:38.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:37 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/648676644' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.194+0000 7f07d01b4640 1 -- 172.21.15.37:0/3976375610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 msgr2=0x7f07c8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.194+0000 7f07d01b4640 1 --2- 172.21.15.37:0/3976375610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c8109320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009ca0 tx=0x7f07c002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- 172.21.15.37:0/3976375610 shutdown_connections 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- 172.21.15.37:0/3976375610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c8109320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- 172.21.15.37:0/3976375610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 0x7f07c81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- 172.21.15.37:0/3976375610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 0x7f07c8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- 172.21.15.37:0/3976375610 >> 172.21.15.37:0/3976375610 conn(0x7f07c80fc460 msgr2=0x7f07c80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- 172.21.15.37:0/3976375610 shutdown_connections 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- 172.21.15.37:0/3976375610 wait complete. 2024-04-25T13:25:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 Processor -- start 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- start start 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 0x7f07c81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 0x7f07c8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810db50 con 0x7f07c8102a00 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810d9d0 con 0x7f07c8102040 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07d01b4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810dcd0 con 0x7f07c8100e60 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.195+0000 7f07ce751640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51644/0 (socket says 172.21.15.37:51644) 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 -- 172.21.15.37:0/1494016881 learned_addr learned my addr 172.21.15.37:0/1494016881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07cdf50640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 0x7f07c81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07cd74f640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 0x7f07c8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 -- 172.21.15.37:0/1494016881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 msgr2=0x7f07c81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 0x7f07c81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 -- 172.21.15.37:0/1494016881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 msgr2=0x7f07c8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 0x7f07c8198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 -- 172.21.15.37:0/1494016881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c819fce0 con 0x7f07c8102a00 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07ce751640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07c0033040 tx=0x7f07c0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00317c0 con 0x7f07c8102a00 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c00044f0 con 0x7f07c8102a00 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c819ff70 con 0x7f07c8102a00 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c002fd00 con 0x7f07c8102a00 2024-04-25T13:25:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.196+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a0480 con 0x7f07c8102a00 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.197+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c8108130 con 0x7f07c8102a00 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.198+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f07c0004040 con 0x7f07c8102a00 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.199+0000 7f07beffd640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 0x7f07b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.199+0000 7f07cdf50640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 0x7f07b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.199+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f07c00f1b90 con 0x7f07c8102a00 2024-04-25T13:25:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.199+0000 7f07cdf50640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 0x7f07b40a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07c40058c0 tx=0x7f07c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.202+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07c00be550 con 0x7f07c8102a00 2024-04-25T13:25:38.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.426+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07c802d5c0 con 0x7f07c8102a00 2024-04-25T13:25:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.427+0000 7f07beffd640 1 -- 172.21.15.37:0/1494016881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7f07c00bde50 con 0x7f07c8102a00 2024-04-25T13:25:38.427 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:38.427 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:25:38.427 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 msgr2=0x7f07b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 0x7f07b40a60f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f07c40058c0 tx=0x7f07c4005850 comp rx=0 tx=0).stop 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 msgr2=0x7f07c819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f07c0033040 tx=0x7f07c0003ba0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 shutdown_connections 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f07b40a3c30 0x7f07b40a60f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07c8102a00 0x7f07c819f670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f07c8102040 0x7f07c8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 --2- 172.21.15.37:0/1494016881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c8100e60 0x7f07c81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 >> 172.21.15.37:0/1494016881 conn(0x7f07c80fc460 msgr2=0x7f07c8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 shutdown_connections 2024-04-25T13:25:38.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.429+0000 7f07d01b4640 1 -- 172.21.15.37:0/1494016881 wait complete. 2024-04-25T13:25:38.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:24:11.206089+0000', 'modified': '2024-04-25T13:24:15.243835+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6839/1356630935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 1356630935}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 1356630935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/1296390680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 1296390680}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 1296390680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24527]} 2024-04-25T13:25:38.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:25:38.441 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 -- 172.21.15.37:0/2709471896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528105460 msgr2=0x7f9528107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 --2- 172.21.15.37:0/2709471896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528105460 0x7f9528107850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9518009900 tx=0x7f951802a410 comp rx=0 tx=0).stop 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 -- 172.21.15.37:0/2709471896 shutdown_connections 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 --2- 172.21.15.37:0/2709471896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528107d90 0x7f952810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 --2- 172.21.15.37:0/2709471896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528105460 0x7f9528107850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 --2- 172.21.15.37:0/2709471896 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528102b30 0x7f9528104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 -- 172.21.15.37:0/2709471896 >> 172.21.15.37:0/2709471896 conn(0x7f95280fc4c0 msgr2=0x7f95280fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 -- 172.21.15.37:0/2709471896 shutdown_connections 2024-04-25T13:25:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.543+0000 7f952e920640 1 -- 172.21.15.37:0/2709471896 wait complete. 2024-04-25T13:25:38.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 Processor -- start 2024-04-25T13:25:38.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 -- start start 2024-04-25T13:25:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528105460 0x7f9528100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 0x7f9528101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f952810db90 con 0x7f9528102b30 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f952810da10 con 0x7f9528105460 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952e920640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f952810dd10 con 0x7f9528107d90 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f9527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f9527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51660/0 (socket says 172.21.15.37:51660) 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f9527fff640 1 -- 172.21.15.37:0/1955552272 learned_addr learned my addr 172.21.15.37:0/1955552272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f952cebd640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 0x7f9528101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.544+0000 7f95277fe640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528105460 0x7f9528100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 -- 172.21.15.37:0/1955552272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 msgr2=0x7f9528101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 0x7f9528101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 -- 172.21.15.37:0/1955552272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528105460 msgr2=0x7f9528100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528105460 0x7f9528100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 -- 172.21.15.37:0/1955552272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95281018b0 con 0x7f9528102b30 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f952cebd640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 0x7f9528101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f9527fff640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c890 tx=0x7f951c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f95257fa640 1 -- 172.21.15.37:0/1955552272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c004220 con 0x7f9528102b30 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f95257fa640 1 -- 172.21.15.37:0/1955552272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f951c0043c0 con 0x7f9528102b30 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95281a23a0 con 0x7f9528102b30 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f95257fa640 1 -- 172.21.15.37:0/1955552272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c004a00 con 0x7f9528102b30 2024-04-25T13:25:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.545+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95281a28c0 con 0x7f9528102b30 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.547+0000 7f95257fa640 1 -- 172.21.15.37:0/1955552272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f951c0079b0 con 0x7f9528102b30 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.547+0000 7f9531636740 1 Processor -- start 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.547+0000 7f9531636740 1 -- start start 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.547+0000 7f9531636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.547+0000 7f9531636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x5646644931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f9531636740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664493710 0x564664499fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f9531636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5646643f6c50 con 0x564664491b80 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f9531636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564664393830 con 0x564664492d50 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f9531636740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646643f2c10 con 0x564664493710 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51666/0 (socket says 172.21.15.37:51666) 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 -- 172.21.15.37:0/1833985411 learned_addr learned my addr 172.21.15.37:0/1833985411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f9527fff640 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x5646644931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f95257fa640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 0x7f95100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f95277fe640 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664493710 0x564664499fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 -- 172.21.15.37:0/1833985411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664493710 msgr2=0x564664499fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664493710 0x564664499fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 -- 172.21.15.37:0/1833985411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 msgr2=0x5646644931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x5646644931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 -- 172.21.15.37:0/1833985411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646641092d0 con 0x564664491b80 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f95257fa640 1 -- 172.21.15.37:0/1955552272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f951c0d02d0 con 0x7f9528102b30 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 0x7f95100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f952cebd640 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f952000e450 tx=0x7f952000e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1115d1d89ab0c542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f950e7fc640 1 -- 172.21.15.37:0/1833985411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520012e60 con 0x564664491b80 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.548+0000 7f950e7fc640 1 -- 172.21.15.37:0/1833985411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952001ce30 con 0x564664491b80 2024-04-25T13:25:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f950e7fc640 1 -- 172.21.15.37:0/1833985411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952001b610 con 0x564664491b80 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- 172.21.15.37:0/1833985411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 msgr2=0x564664491fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f952cebd640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 0x7f95100a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f952001f000 tx=0x7f9520020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f952000e450 tx=0x7f952000e910 comp rx=0 tx=0).stop 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- 172.21.15.37:0/1833985411 shutdown_connections 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664493710 0x564664499fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x5646644931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 --2- 172.21.15.37:0/1833985411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x564664491fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- 172.21.15.37:0/1833985411 >> 172.21.15.37:0/1833985411 conn(0x56466448d070 msgr2=0x56466448d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- 172.21.15.37:0/1833985411 shutdown_connections 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- 172.21.15.37:0/1833985411 wait complete. 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.549+0000 7f9531636740 1 -- start start 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 Processor -- start 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x56466449c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56466449a720 con 0x564664493710 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564664391470 con 0x564664492d50 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56466449a8a0 con 0x564664491b80 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9531636740 10 client.0 _ll_register_callbacks cb 0x5646643b6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51668/0 (socket says 172.21.15.37:51668) 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 -- 172.21.15.37:0/3916134465 learned_addr learned my addr 172.21.15.37:0/3916134465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9527fff640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 -- 172.21.15.37:0/3916134465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 msgr2=0x56466449bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 -- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 msgr2=0x56466449c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f95277fe640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x56466449c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x56466449c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f952cebd640 1 -- 172.21.15.37:0/3916134465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646641d36b0 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.550+0000 7f9527fff640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f952cebd640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9520012550 tx=0x7f9520024a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98926ba4876ab318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9520034020 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f952001b5d0 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f952001ce30 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f95200126d0 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646644a6890 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952001ce30 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9520025710 con 0x564664493710 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:25:38.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 20 client.15054 populate_metadata read hostname 'smithi037' 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646644a8c00 0x5646644aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 4 client.15054 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.551+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 --> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564664404670 con 0x5646644a8c00 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.552+0000 7f9527fff640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646644a8c00 0x5646644aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.552+0000 7f9527fff640 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646644a8c00 0x5646644aa070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88d0389e1955df9 server_cookie=f95748e7bd8ee54 in_seq=0 out_seq=0 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.552+0000 7f95077fe640 10 client.15054 ms_handle_connect on v2:172.21.15.44:6838/1296390680 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f95077fe640 1 -- 172.21.15.37:0/3916134465 <== mds.0 v2:172.21.15.44:6838/1296390680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564664404670 con 0x5646644a8c00 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f95077fe640 10 client.15054 handle_command_reply: tid=0 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 client.15054 shutdown 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 msgr2=0x5646644a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9520012550 tx=0x7f9520024a50 comp rx=0 tx=0).stop 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 shutdown_connections 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646644a8c00 0x5646644aa070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664493710 0x5646644a3520 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664492d50 0x56466449c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 --2- 172.21.15.37:0/3916134465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f95077fe640 0 client.15054 ms_handle_reset on v2:172.21.15.44:6838/1296390680 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 >> 172.21.15.37:0/3916134465 conn(0x56466448d070 msgr2=0x56466448c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 shutdown_connections 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 1 -- 172.21.15.37:0/3916134465 wait complete. 2024-04-25T13:25:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.553+0000 7f9531636740 20 client.15054 trim_cache size 0 max 16384 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.571+0000 7f9531636740 1 Processor -- start 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 -- start start 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x56466449b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5646644a83b0 con 0x564664492d50 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5646644a8a80 con 0x564664493710 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f9531636740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646644a9150 con 0x564664491b80 2024-04-25T13:25:38.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51682/0 (socket says 172.21.15.37:51682) 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 -- 172.21.15.37:0/2449326428 learned_addr learned my addr 172.21.15.37:0/2449326428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f952cebd640 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x56466449b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 -- 172.21.15.37:0/2449326428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 msgr2=0x56466449ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 -- 172.21.15.37:0/2449326428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 msgr2=0x56466449b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x56466449b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 -- 172.21.15.37:0/2449326428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466448cea0 con 0x564664492d50 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f952cebd640 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x56466449b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f95277fe640 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9528069d50 tx=0x7f9518032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4213b70eb060a011 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f950dffb640 1 -- 172.21.15.37:0/2449326428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9518035070 con 0x564664492d50 2024-04-25T13:25:38.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f950dffb640 1 -- 172.21.15.37:0/2449326428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9518032e50 con 0x564664492d50 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.572+0000 7f950dffb640 1 -- 172.21.15.37:0/2449326428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95180046c0 con 0x564664492d50 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- 172.21.15.37:0/2449326428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 msgr2=0x5646644a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9528069d50 tx=0x7f9518032d60 comp rx=0 tx=0).stop 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- 172.21.15.37:0/2449326428 shutdown_connections 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x56466449b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664492d50 0x5646644a34e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 --2- 172.21.15.37:0/2449326428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664491b80 0x56466449ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- 172.21.15.37:0/2449326428 >> 172.21.15.37:0/2449326428 conn(0x56466448d070 msgr2=0x56466449bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- 172.21.15.37:0/2449326428 shutdown_connections 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- 172.21.15.37:0/2449326428 wait complete. 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 -- start start 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.573+0000 7f9531636740 1 Processor -- start 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664492d50 0x56466449f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x5646644a2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5646644a9ef0 con 0x564664491b80 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5646644a9820 con 0x564664493710 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646644aa5c0 con 0x564664492d50 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 10 client.0 _ll_register_callbacks cb 0x7f9520016050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51694/0 (socket says 172.21.15.37:51694) 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 -- 172.21.15.37:0/1088753783 learned_addr learned my addr 172.21.15.37:0/1088753783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f95277fe640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664492d50 0x56466449f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 -- 172.21.15.37:0/1088753783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664492d50 msgr2=0x56466449f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f952cebd640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x5646644a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664492d50 0x56466449f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 -- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 msgr2=0x5646644a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x5646644a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 -- 172.21.15.37:0/1088753783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646644a2940 con 0x564664491b80 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9527fff640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f951c098990 tx=0x7f951c09bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d9266a3991989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f95077fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c01d040 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f951c007ca0 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9518032e50 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.574+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f951c09c530 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 10 client.15060 fetch_fsmap learned FSMap version 20 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56466450ece0 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9518032e50 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f951c098280 con 0x564664491b80 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 10 client.15060 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 20 client.15060 populate_metadata read hostname 'smithi037' 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646645138b0 0x5646645125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 4 client.15060 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 --> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] -- command(tid 0: {"prefix": "status"}) -- 0x5646643fe4e0 con 0x5646645138b0 2024-04-25T13:25:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.575+0000 7f95277fe640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646645138b0 0x5646645125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f95077fe640 10 client.15060 ms_handle_connect on v2:172.21.15.44:6838/1296390680 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f95277fe640 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646645138b0 0x5646645125c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5db319d9f0d6 server_cookie=ddff5553547bd8d7 in_seq=0 out_seq=0 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f95077fe640 1 -- 172.21.15.37:0/1088753783 <== mds.0 v2:172.21.15.44:6838/1296390680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5646643fe4e0 con 0x5646645138b0 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f95077fe640 10 client.15060 handle_command_reply: tid=0 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 client.15060 shutdown 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 msgr2=0x56466449edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f951c098990 tx=0x7f951c09bf50 comp rx=0 tx=0).stop 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 shutdown_connections 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5646645138b0 0x5646645125c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564664493710 0x5646644a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564664492d50 0x56466449f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f95077fe640 0 client.15060 ms_handle_reset on v2:172.21.15.44:6838/1296390680 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 --2- 172.21.15.37:0/1088753783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564664491b80 0x56466449edf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 >> 172.21.15.37:0/1088753783 conn(0x56466448d070 msgr2=0x56466449ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 shutdown_connections 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 1 -- 172.21.15.37:0/1088753783 wait complete. 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.576+0000 7f9531636740 20 client.15060 trim_cache size 0 max 16384 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24527, 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T13:25:38.583 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 86.707585901000002, 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 20, 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 91.926662265999994 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:25:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 msgr2=0x7f95100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 0x7f95100a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f952001f000 tx=0x7f9520020000 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 msgr2=0x7f95281024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f951c00c890 tx=0x7f951c00cd50 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 shutdown_connections 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9528107d90 0x7f9528101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9528105460 0x7f9528100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f95100a3bb0 0x7f95100a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 --2- 172.21.15.37:0/1955552272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9528102b30 0x7f95281024a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 >> 172.21.15.37:0/1955552272 conn(0x7f95280fc4c0 msgr2=0x7f95281033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 shutdown_connections 2024-04-25T13:25:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:38.577+0000 7f952e920640 1 -- 172.21.15.37:0/1955552272 wait complete. 2024-04-25T13:25:38.588 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T13:25:38.588 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:25:38.598 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:25:38.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T13:25:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:38 smithi037 ceph-mon[29204]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:38 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/676627953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:38 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1494016881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:38 smithi069 ceph-mon[32534]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:38 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/676627953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:38 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1494016881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:38 smithi044 ceph-mon[32590]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:38 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/676627953' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:25:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:38 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1494016881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:25:40.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 -- 172.21.15.37:0/1971800270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39281019a0 msgr2=0x7f3928101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 --2- 172.21.15.37:0/1971800270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39281019a0 0x7f3928101dc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f391c0099c0 tx=0x7f391c009cd0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 -- 172.21.15.37:0/1971800270 shutdown_connections 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 --2- 172.21.15.37:0/1971800270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 0x7f3928109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 --2- 172.21.15.37:0/1971800270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39281019a0 0x7f3928101dc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 --2- 172.21.15.37:0/1971800270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3928102ba0 0x7f3928103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 -- 172.21.15.37:0/1971800270 >> 172.21.15.37:0/1971800270 conn(0x7f39280fcf40 msgr2=0x7f39280ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 -- 172.21.15.37:0/1971800270 shutdown_connections 2024-04-25T13:25:40.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.525+0000 7f392c872700 1 -- 172.21.15.37:0/1971800270 wait complete. 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 Processor -- start 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 -- start start 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39281019a0 0x7f3928198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 0x7f39281a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f392810d410 con 0x7f3928102ba0 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f392810d290 con 0x7f39281019a0 2024-04-25T13:25:40.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392c872700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f392810d590 con 0x7f3928103540 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35676/0 (socket says 172.21.15.37:35676) 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 -- 172.21.15.37:0/1396927991 learned_addr learned my addr 172.21.15.37:0/1396927991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3926d9d700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 0x7f39281a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f392659c700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39281019a0 0x7f3928198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 -- 172.21.15.37:0/1396927991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 msgr2=0x7f39281a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 0x7f39281a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 -- 172.21.15.37:0/1396927991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39281019a0 msgr2=0x7f3928198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39281019a0 0x7f3928198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 -- 172.21.15.37:0/1396927991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39281a0840 con 0x7f3928102ba0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.526+0000 7f3925d9b700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f391800d980 tx=0x7f391800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.527+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918016070 con 0x7f3928102ba0 2024-04-25T13:25:40.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.527+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39281a0b30 con 0x7f3928102ba0 2024-04-25T13:25:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.527+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f391800de70 con 0x7f3928102ba0 2024-04-25T13:25:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.527+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800f3c0 con 0x7f3928102ba0 2024-04-25T13:25:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.527+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39281a10c0 con 0x7f3928102ba0 2024-04-25T13:25:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.528+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3928069730 con 0x7f3928102ba0 2024-04-25T13:25:40.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.529+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3918019070 con 0x7f3928102ba0 2024-04-25T13:25:40.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.529+0000 7f39177fe700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 0x7f39100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:40.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.530+0000 7f392659c700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 0x7f39100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:40.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.530+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f39180d02b0 con 0x7f3928102ba0 2024-04-25T13:25:40.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.530+0000 7f392659c700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 0x7f39100a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f391c00b580 tx=0x7f391c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:40.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.534+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f391809cc70 con 0x7f3928102ba0 2024-04-25T13:25:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.760+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f392806b880 con 0x7f39100a3c40 2024-04-25T13:25:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.761+0000 7f39177fe700 1 -- 172.21.15.37:0/1396927991 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f392806b880 con 0x7f39100a3c40 2024-04-25T13:25:40.761 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-04-25T13:25:40.762 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-04-25T13:25:40.762 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 msgr2=0x7f39100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 0x7f39100a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f391c00b580 tx=0x7f391c005fb0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 msgr2=0x7f3928199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f391800d980 tx=0x7f391800dc90 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 shutdown_connections 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f39100a3c40 0x7f39100a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39281019a0 0x7f3928198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3928103540 0x7f39281a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 --2- 172.21.15.37:0/1396927991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3928102ba0 0x7f3928199170 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 >> 172.21.15.37:0/1396927991 conn(0x7f39280fcf40 msgr2=0x7f392806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 shutdown_connections 2024-04-25T13:25:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:40.763+0000 7f392c872700 1 -- 172.21.15.37:0/1396927991 wait complete. 2024-04-25T13:25:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:40 smithi037 ceph-mon[29204]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:40 smithi069 ceph-mon[32534]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:40 smithi044 ceph-mon[32590]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:41.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T13:25:41.696 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:42.016 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:41 smithi037 ceph-mon[29204]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:42.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:41 smithi069 ceph-mon[32534]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:42.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:41 smithi044 ceph-mon[32590]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:43.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:42 smithi037 ceph-mon[29204]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:42 smithi069 ceph-mon[32534]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:42 smithi044 ceph-mon[32590]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:43.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 -- 172.21.15.37:0/2688566103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c104670 msgr2=0x7f6a5c10af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 --2- 172.21.15.37:0/2688566103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c104670 0x7f6a5c10af20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b0d0 tx=0x7f6a5400b3e0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 -- 172.21.15.37:0/2688566103 shutdown_connections 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 --2- 172.21.15.37:0/2688566103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c102ad0 0x7f6a5c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 --2- 172.21.15.37:0/2688566103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c104670 0x7f6a5c10af20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 --2- 172.21.15.37:0/2688566103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 -- 172.21.15.37:0/2688566103 >> 172.21.15.37:0/2688566103 conn(0x7f6a5c0fe090 msgr2=0x7f6a5c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 -- 172.21.15.37:0/2688566103 shutdown_connections 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 -- 172.21.15.37:0/2688566103 wait complete. 2024-04-25T13:25:43.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.319+0000 7f6a6218f700 1 Processor -- start 2024-04-25T13:25:43.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 -- start start 2024-04-25T13:25:43.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c102ad0 0x7f6a5c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:43.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c104670 0x7f6a5c1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10e550 con 0x7f6a5c103cd0 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10e3d0 con 0x7f6a5c104670 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a6218f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c10e6d0 con 0x7f6a5c102ad0 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35680/0 (socket says 172.21.15.37:35680) 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 -- 172.21.15.37:0/356224714 learned_addr learned my addr 172.21.15.37:0/356224714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5bfff700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c104670 0x7f6a5c1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5b7fe700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c102ad0 0x7f6a5c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 -- 172.21.15.37:0/356224714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c102ad0 msgr2=0x7f6a5c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:43.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c102ad0 0x7f6a5c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 -- 172.21.15.37:0/356224714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c104670 msgr2=0x7f6a5c1a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c104670 0x7f6a5c1a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 -- 172.21.15.37:0/356224714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c1a0700 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.320+0000 7f6a5affd700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c950 tx=0x7f6a5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.321+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50007ac0 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.321+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a50004ce0 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.321+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c1a09f0 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.321+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50005580 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.321+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1a0f30 con 0x7f6a5c103cd0 2024-04-25T13:25:43.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.322+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c069730 con 0x7f6a5c103cd0 2024-04-25T13:25:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.323+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6a50007c60 con 0x7f6a5c103cd0 2024-04-25T13:25:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.323+0000 7f6a58ff9700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 0x7f6a440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.324+0000 7f6a5b7fe700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 0x7f6a440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.324+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f6a500d03d0 con 0x7f6a5c103cd0 2024-04-25T13:25:43.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.324+0000 7f6a5b7fe700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 0x7f6a440a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005fd0 tx=0x7f6a4c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:43.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.327+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a50014030 con 0x7f6a5c103cd0 2024-04-25T13:25:43.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.546+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6a5c062e80 con 0x7f6a440a3c40 2024-04-25T13:25:43.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.560+0000 7f6a58ff9700 1 -- 172.21.15.37:0/356224714 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f6a5c062e80 con 0x7f6a440a3c40 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (100s) 15s ago 99s 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9c8e8392d2e1 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi044 running (97s) 31s ago 96s 25.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 55d02c9377c3 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi069 running (111s) 31s ago 111s 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7502ab2f9297 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (119s) 15s ago 119s 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e06315133780 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi044 running (117s) 31s ago 116s 15.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 600bdc8f5abe 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi069 running (114s) 31s ago 113s 13.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b2048e2795f1 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (103s) 15s ago 102s 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 630ddbb5eadf 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi044 running (108s) 31s ago 108s 12.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 98d1ca477f87 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi069 running (105s) 31s ago 105s 17.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 54eeb09fb9c6 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (2m) 15s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 305be8b65f17 2024-04-25T13:25:43.561 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi044 running (2m) 31s ago 2m 12.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c26924caa949 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi069 running (2m) 31s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fc1337416794 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (9m) 15s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4773bce144a7 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi044 *:8443,8765 running (7m) 31s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 554bc2fe87fc 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi069 *:8443,8765 running (7m) 31s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2344106ab1fe 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (9m) 15s ago 9m 52.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 d16148f0b3ad 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi044 running (7m) 31s ago 7m 41.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 fff21b0440d5 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi069 running (7m) 31s ago 7m 38.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 98847380f4eb 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 15s ago 7m 56.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 c5279eb86b41 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi044 running (6m) 31s ago 6m 56.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b17c45403b9b 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi069 running (6m) 31s ago 6m 60.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 354de7c521c8 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (5m) 15s ago 5m 55.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 2451519dd283 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi044 running (5m) 31s ago 5m 52.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 06c74657362f 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi069 running (5m) 31s ago 5m 46.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 93717c685db8 2024-04-25T13:25:43.562 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (4m) 15s ago 4m 51.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 10d3cf795b91 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi044 running (4m) 31s ago 4m 57.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 936c3a2f65b9 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi069 running (3m) 31s ago 3m 55.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5c69cacbe80c 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (3m) 15s ago 3m 51.8M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 b5e0a3a41773 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi044 running (2m) 31s ago 2m 54.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 a53122d54c3f 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi069 running (2m) 31s ago 2m 52.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b3baf1d3614a 2024-04-25T13:25:43.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 msgr2=0x7f6a440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 0x7f6a440a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005fd0 tx=0x7f6a4c005e50 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 msgr2=0x7f6a5c198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c950 tx=0x7f6a5000cc60 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 shutdown_connections 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f6a440a3c40 0x7f6a440a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a5c104670 0x7f6a5c1a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a5c102ad0 0x7f6a5c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 --2- 172.21.15.37:0/356224714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c103cd0 0x7f6a5c198fe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 >> 172.21.15.37:0/356224714 conn(0x7f6a5c0fe090 msgr2=0x7f6a5c106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 shutdown_connections 2024-04-25T13:25:43.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:43.563+0000 7f6a6218f700 1 -- 172.21.15.37:0/356224714 wait complete. 2024-04-25T13:25:44.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:43 smithi037 ceph-mon[29204]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:44.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:43 smithi069 ceph-mon[32534]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:44.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:43 smithi044 ceph-mon[32590]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:44.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T13:25:44.533 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:44.913 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:44 smithi037 ceph-mon[29204]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:44 smithi069 ceph-mon[32534]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:45.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:44 smithi044 ceph-mon[32590]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 -- 172.21.15.37:0/3660982406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103cc0 msgr2=0x7f99f4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 --2- 172.21.15.37:0/3660982406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103cc0 0x7f99f4104120 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99e40099c0 tx=0x7f99e4009cd0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 -- 172.21.15.37:0/3660982406 shutdown_connections 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 --2- 172.21.15.37:0/3660982406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4104660 0x7f99f410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 --2- 172.21.15.37:0/3660982406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4102ac0 0x7f99f4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 --2- 172.21.15.37:0/3660982406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4103cc0 0x7f99f4104120 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 -- 172.21.15.37:0/3660982406 >> 172.21.15.37:0/3660982406 conn(0x7f99f40fe060 msgr2=0x7f99f41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 -- 172.21.15.37:0/3660982406 shutdown_connections 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.263+0000 7f99fbd57700 1 -- 172.21.15.37:0/3660982406 wait complete. 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 Processor -- start 2024-04-25T13:25:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 -- start start 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4102ac0 0x7f99f419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 0x7f99f419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410e550 con 0x7f99f4104660 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410e3d0 con 0x7f99f4102ac0 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fbd57700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410e6d0 con 0x7f99f4103cc0 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35690/0 (socket says 172.21.15.37:35690) 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 -- 172.21.15.37:0/218824531 learned_addr learned my addr 172.21.15.37:0/218824531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99f9af3700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4102ac0 0x7f99f419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99f92f2700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 0x7f99f419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 -- 172.21.15.37:0/218824531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 msgr2=0x7f99f419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 0x7f99f419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 -- 172.21.15.37:0/218824531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4102ac0 msgr2=0x7f99f419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4102ac0 0x7f99f419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99fa2f4700 1 -- 172.21.15.37:0/218824531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f41a2a70 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.264+0000 7f99f92f2700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 0x7f99f419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99fa2f4700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ef60 tx=0x7f99ec00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0092e0 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec004bb0 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004280 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f41a2d00 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.265+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f41a31e0 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.266+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f4069760 con 0x7f99f4104660 2024-04-25T13:25:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.267+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99ec01b030 con 0x7f99f4104660 2024-04-25T13:25:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.268+0000 7f99eaffd700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 0x7f99e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.268+0000 7f99f9af3700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 0x7f99e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.268+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f99ec0d0830 con 0x7f99f4104660 2024-04-25T13:25:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.269+0000 7f99f9af3700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 0x7f99e00a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99f4103b20 tx=0x7f99f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.272+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99f4069760 con 0x7f99f4104660 2024-04-25T13:25:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.492+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f99f4076390 con 0x7f99e00a3cc0 2024-04-25T13:25:46.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.498+0000 7f99eaffd700 1 -- 172.21.15.37:0/218824531 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f99f4076390 con 0x7f99e00a3cc0 2024-04-25T13:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T13:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 33s ago 2m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi044=b;smithi044=e;smithi044=h;smithi044=k;smithi069=c;smithi069=f;smithi069=i;smithi069=l;count:12 2024-04-25T13:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 33s ago 7m smithi037=x;smithi044=y;smithi069=z;count:3 2024-04-25T13:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 33s ago 7m smithi037:172.21.15.37=a;smithi044:172.21.15.44=b;smithi069:172.21.15.69=c;count:3 2024-04-25T13:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 33s ago - 2024-04-25T13:25:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 msgr2=0x7f99e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 0x7f99e00a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99f4103b20 tx=0x7f99f000b410 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 msgr2=0x7f99f41a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ef60 tx=0x7f99ec00ef90 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 shutdown_connections 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f99e00a3cc0 0x7f99e00a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99f4102ac0 0x7f99f419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f4104660 0x7f99f41a23d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 --2- 172.21.15.37:0/218824531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f99f4103cc0 0x7f99f419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 >> 172.21.15.37:0/218824531 conn(0x7f99f40fe060 msgr2=0x7f99f4074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 shutdown_connections 2024-04-25T13:25:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:46.500+0000 7f99fbd57700 1 -- 172.21.15.37:0/218824531 wait complete. 2024-04-25T13:25:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:46 smithi037 ceph-mon[29204]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:46 smithi037 ceph-mon[29204]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:46 smithi069 ceph-mon[32534]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:46 smithi069 ceph-mon[32534]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:46 smithi044 ceph-mon[32590]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:46 smithi044 ceph-mon[32590]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:47.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T13:25:47.587 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:48 smithi037 ceph-mon[29204]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:48 smithi069 ceph-mon[32534]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:48 smithi044 ceph-mon[32590]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1863721444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 msgr2=0x7f0c40109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1863721444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c40109df0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800b0d0 tx=0x7f0c3800b3e0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1863721444 shutdown_connections 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1863721444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c401019a0 0x7f0c40101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1863721444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c40109df0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1863721444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102ba0 0x7f0c40103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1863721444 >> 172.21.15.37:0/1863721444 conn(0x7f0c400fcf40 msgr2=0x7f0c400ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:49.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1863721444 shutdown_connections 2024-04-25T13:25:49.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.428+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1863721444 wait complete. 2024-04-25T13:25:49.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 Processor -- start 2024-04-25T13:25:49.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 -- start start 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c401019a0 0x7f0c40105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 0x7f0c40103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4010d440 con 0x7f0c40103540 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4010d2c0 con 0x7f0c40102ba0 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.429+0000 7f0c44ba5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4010d5c0 con 0x7f0c401019a0 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35702/0 (socket says 172.21.15.37:35702) 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 -- 172.21.15.37:0/1378213438 learned_addr learned my addr 172.21.15.37:0/1378213438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3dd9b700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 0x7f0c40103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3e59c700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c401019a0 0x7f0c40105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 -- 172.21.15.37:0/1378213438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c401019a0 msgr2=0x7f0c40105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:49.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c401019a0 0x7f0c40105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 -- 172.21.15.37:0/1378213438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 msgr2=0x7f0c40103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 0x7f0c40103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 -- 172.21.15.37:0/1378213438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c40104d70 con 0x7f0c40103540 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3dd9b700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 0x7f0c40103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c3ed9d700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009cd0 tx=0x7f0c38007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3801f070 con 0x7f0c40103540 2024-04-25T13:25:49.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c401a2d80 con 0x7f0c40103540 2024-04-25T13:25:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c38003e10 con 0x7f0c40103540 2024-04-25T13:25:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3801b670 con 0x7f0c40103540 2024-04-25T13:25:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.430+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a3180 con 0x7f0c40103540 2024-04-25T13:25:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.432+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c40069730 con 0x7f0c40103540 2024-04-25T13:25:49.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.432+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c38005050 con 0x7f0c40103540 2024-04-25T13:25:49.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.433+0000 7f0c2f7fe700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 0x7f0c280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:49.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.433+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f0c3800f020 con 0x7f0c40103540 2024-04-25T13:25:49.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.433+0000 7f0c3e59c700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 0x7f0c280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:49.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.434+0000 7f0c3e59c700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 0x7f0c280a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c40102a00 tx=0x7f0c34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:49.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.438+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c3809dc40 con 0x7f0c40103540 2024-04-25T13:25:49.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.650+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0c400ff130 con 0x7f0c280a3cc0 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.652+0000 7f0c2f7fe700 1 -- 172.21.15.37:0/1378213438 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f0c400ff130 con 0x7f0c280a3cc0 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stdout:smithi044 172.21.15.44 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stdout:smithi069 172.21.15.69 2024-04-25T13:25:49.652 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-04-25T13:25:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.653+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 msgr2=0x7f0c280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.653+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 0x7f0c280a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c40102a00 tx=0x7f0c34009260 comp rx=0 tx=0).stop 2024-04-25T13:25:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.653+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 msgr2=0x7f0c401044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009cd0 tx=0x7f0c38007ad0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 shutdown_connections 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0c280a3cc0 0x7f0c280a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40103540 0x7f0c401044c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c401019a0 0x7f0c40105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 --2- 172.21.15.37:0/1378213438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c40102ba0 0x7f0c40103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 >> 172.21.15.37:0/1378213438 conn(0x7f0c400fcf40 msgr2=0x7f0c400fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 shutdown_connections 2024-04-25T13:25:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:49.654+0000 7f0c44ba5700 1 -- 172.21.15.37:0/1378213438 wait complete. 2024-04-25T13:25:50.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T13:25:50.753 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:51.093 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:50 smithi037 ceph-mon[29204]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:51.093 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:50 smithi037 ceph-mon[29204]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:50 smithi069 ceph-mon[32534]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:50 smithi069 ceph-mon[32534]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:50 smithi044 ceph-mon[32590]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:51.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:50 smithi044 ceph-mon[32590]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 -- 172.21.15.37:0/1967027710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890103cc0 msgr2=0x7f1890104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 --2- 172.21.15.37:0/1967027710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890103cc0 0x7f1890104120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f18800099c0 tx=0x7f1880009cd0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 -- 172.21.15.37:0/1967027710 shutdown_connections 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 --2- 172.21.15.37:0/1967027710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890104660 0x7f189010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 --2- 172.21.15.37:0/1967027710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890102ac0 0x7f1890102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 --2- 172.21.15.37:0/1967027710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890103cc0 0x7f1890104120 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 -- 172.21.15.37:0/1967027710 >> 172.21.15.37:0/1967027710 conn(0x7f18900fe060 msgr2=0x7f18901004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 -- 172.21.15.37:0/1967027710 shutdown_connections 2024-04-25T13:25:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.535+0000 7f1895bc1700 1 -- 172.21.15.37:0/1967027710 wait complete. 2024-04-25T13:25:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 Processor -- start 2024-04-25T13:25:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 -- start start 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 0x7f1890199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f189010e530 con 0x7f1890104660 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f189010e3b0 con 0x7f1890102ac0 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f1895bc1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f189010e6b0 con 0x7f1890103cc0 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49556/0 (socket says 172.21.15.37:49556) 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188f7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:56838/0 (socket says 172.21.15.37:56838) 2024-04-25T13:25:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188ffff700 1 -- 172.21.15.37:0/578639134 learned_addr learned my addr 172.21.15.37:0/578639134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.536+0000 7f188effd700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 0x7f1890199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 -- 172.21.15.37:0/578639134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 msgr2=0x7f1890199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 0x7f1890199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 -- 172.21.15.37:0/578639134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 msgr2=0x7f1890198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 -- 172.21.15.37:0/578639134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18901a0780 con 0x7f1890104660 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188effd700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 0x7f1890199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188f7fe700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188ffff700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f188800ef60 tx=0x7f188800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18880092e0 con 0x7f1890104660 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1888004bb0 con 0x7f1890104660 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888004280 con 0x7f1890104660 2024-04-25T13:25:52.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18901a0a10 con 0x7f1890104660 2024-04-25T13:25:52.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.537+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901a0f00 con 0x7f1890104660 2024-04-25T13:25:52.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.538+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890069730 con 0x7f1890104660 2024-04-25T13:25:52.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.540+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f188801b030 con 0x7f1890104660 2024-04-25T13:25:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.540+0000 7f188cff9700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 0x7f18780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.541+0000 7f188f7fe700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 0x7f18780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.541+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f18880d0860 con 0x7f1890104660 2024-04-25T13:25:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.541+0000 7f188f7fe700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 0x7f18780a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1890103b20 tx=0x7f188400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.544+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1888022020 con 0x7f1890104660 2024-04-25T13:25:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.759+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1890062e80 con 0x7f18780a3cc0 2024-04-25T13:25:52.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.761+0000 7f188cff9700 1 -- 172.21.15.37:0/578639134 <== mgr.14152 v2:172.21.15.37:6800/2819711067 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1890062e80 con 0x7f18780a3cc0 2024-04-25T13:25:52.762 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T13:25:52.762 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T13:25:52.762 INFO:teuthology.orchestra.run.smithi037.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:25:52.762 INFO:teuthology.orchestra.run.smithi037.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 msgr2=0x7f18780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 0x7f18780a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1890103b20 tx=0x7f188400a3b0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 msgr2=0x7f18901a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f188800ef60 tx=0x7f188800ef90 comp rx=0 tx=0).stop 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 shutdown_connections 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f18780a3cc0 0x7f18780a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1890102ac0 0x7f1890198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1890104660 0x7f18901a00e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 --2- 172.21.15.37:0/578639134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1890103cc0 0x7f1890199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:52.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.763+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 >> 172.21.15.37:0/578639134 conn(0x7f18900fe060 msgr2=0x7f1890106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:52.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.764+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 shutdown_connections 2024-04-25T13:25:52.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:52.764+0000 7f1895bc1700 1 -- 172.21.15.37:0/578639134 wait complete. 2024-04-25T13:25:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:52 smithi037 ceph-mon[29204]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:52 smithi069 ceph-mon[32534]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:52 smithi044 ceph-mon[32590]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:25:53.482 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T13:25:53.491 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:25:53.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T13:25:53.752 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:54.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:53 smithi037 ceph-mon[29204]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:54.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:53 smithi069 ceph-mon[32534]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:54.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:53 smithi044 ceph-mon[32590]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:25:54.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:54 smithi037 ceph-mon[29204]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:54 smithi069 ceph-mon[32534]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:54 smithi044 ceph-mon[32590]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:55.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 -- 172.21.15.37:0/32630758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0102ba0 msgr2=0x7fbcd0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:55.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/32630758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0102ba0 0x7fbcd0103000 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00099c0 tx=0x7fbcc0009cd0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 -- 172.21.15.37:0/32630758 shutdown_connections 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/32630758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcd01019a0 0x7fbcd0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/32630758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 0x7fbcd0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/32630758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd0102ba0 0x7fbcd0103000 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 -- 172.21.15.37:0/32630758 >> 172.21.15.37:0/32630758 conn(0x7fbcd00fcf40 msgr2=0x7fbcd00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 -- 172.21.15.37:0/32630758 shutdown_connections 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.422+0000 7fbcd87c8700 1 -- 172.21.15.37:0/32630758 wait complete. 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 Processor -- start 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 -- start start 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcd0102ba0 0x7fbcd0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 0x7fbcd01a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:55.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010d4b0 con 0x7fbcd01019a0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010d330 con 0x7fbcd0102ba0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd87c8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010d630 con 0x7fbcd0103540 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49570/0 (socket says 172.21.15.37:49570) 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 -- 172.21.15.37:0/1313423031 learned_addr learned my addr 172.21.15.37:0/1313423031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6d65700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 0x7fbcd01a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 -- 172.21.15.37:0/1313423031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 msgr2=0x7fbcd01a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 0x7fbcd01a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 -- 172.21.15.37:0/1313423031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcd0102ba0 msgr2=0x7fbcd0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcd0102ba0 0x7fbcd0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.423+0000 7fbcd6564700 1 -- 172.21.15.37:0/1313423031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd01a0930 con 0x7fbcd01019a0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcd6564700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00b500 tx=0x7fbccc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:55.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc015020 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbccc011ce0 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd01a0c20 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc010d60 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.424+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd01a1160 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.425+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd0069730 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.427+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbccc018020 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.427+0000 7fbcc77fe700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 0x7fbcbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.427+0000 7fbcd5d63700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 0x7fbcbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.427+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fbccc0d0200 con 0x7fbcd01019a0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.428+0000 7fbcd5d63700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 0x7fbcbc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00098e0 tx=0x7fbcc0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:55.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.431+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbccc01f020 con 0x7fbcd01019a0 2024-04-25T13:25:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.689+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbcd01a1390 con 0x7fbcd01019a0 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.689+0000 7fbcc77fe700 1 -- 172.21.15.37:0/1313423031 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+2998 (secure 0 0 0) 0x7fbcd01a1390 con 0x7fbcd01019a0 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:e20 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 20 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T13:25:55.690 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T13:24:11.206089+0000 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-25T13:24:15.243835+0000 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 1 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:in 0 2024-04-25T13:25:55.691 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24527} 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24527 members: 24527 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24527} state up:active seq 3 addr [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{0:24520} state up:standby-replay seq 1 addr [v2:172.21.15.69:6838/1356630935,v1:172.21.15.69:6839/1356630935] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.692 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.37:6834/1814974020,v1:172.21.15.37:6835/1814974020] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.37:6836/3506634050,v1:172.21.15.37:6837/3506634050] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14631} state up:standby seq 1 addr [v2:172.21.15.37:6838/3019792631,v1:172.21.15.37:6839/3019792631] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14643} state up:standby seq 1 addr [v2:172.21.15.37:6840/1127100718,v1:172.21.15.37:6841/1127100718] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24422} state up:standby seq 1 addr [v2:172.21.15.44:6832/2542542028,v1:172.21.15.44:6833/2542542028] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.69:6832/579371697,v1:172.21.15.69:6833/579371697] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{-1:24455} state up:standby seq 1 addr [v2:172.21.15.44:6834/1266567676,v1:172.21.15.44:6835/1266567676] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{-1:24482} state up:standby seq 1 addr [v2:172.21.15.44:6836/690443752,v1:172.21.15.44:6837/690443752] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{-1:24484} state up:standby seq 1 addr [v2:172.21.15.69:6834/2452024894,v1:172.21.15.69:6835/2452024894] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.693 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{-1:24496} state up:standby seq 1 addr [v2:172.21.15.69:6836/2736481323,v1:172.21.15.69:6837/2736481323] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T13:25:55.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 msgr2=0x7fbcbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:55.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 0x7fbcbc0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00098e0 tx=0x7fbcc0009f10 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 msgr2=0x7fbcd0198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00b500 tx=0x7fbccc00b8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 shutdown_connections 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fbcbc0a3c40 0x7fbcbc0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcd0103540 0x7fbcd01a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd01019a0 0x7fbcd0198ce0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 --2- 172.21.15.37:0/1313423031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbcd0102ba0 0x7fbcd0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 >> 172.21.15.37:0/1313423031 conn(0x7fbcd00fcf40 msgr2=0x7fbcd006a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 shutdown_connections 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:55.691+0000 7fbcd87c8700 1 -- 172.21.15.37:0/1313423031 wait complete. 2024-04-25T13:25:55.695 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:25:56.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:55 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1313423031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:25:56.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:55 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1313423031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:25:56.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:55 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1313423031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T13:25:56.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T13:25:56.631 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:25:57.064 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:56 smithi037 ceph-mon[29204]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:56 smithi069 ceph-mon[32534]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:56 smithi044 ceph-mon[32590]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:25:58.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 -- 172.21.15.37:0/1910012660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 msgr2=0x7f7424102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:58.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 --2- 172.21.15.37:0/1910012660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f7424102ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f74200099c0 tx=0x7f7420009cd0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 -- 172.21.15.37:0/1910012660 shutdown_connections 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 --2- 172.21.15.37:0/1910012660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424104640 0x7f742410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 --2- 172.21.15.37:0/1910012660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f7424102ec0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 --2- 172.21.15.37:0/1910012660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7424103ca0 0x7f7424104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 -- 172.21.15.37:0/1910012660 >> 172.21.15.37:0/1910012660 conn(0x7f74240fe040 msgr2=0x7f7424100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 -- 172.21.15.37:0/1910012660 shutdown_connections 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.439+0000 7f742b93f700 1 -- 172.21.15.37:0/1910012660 wait complete. 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 Processor -- start 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 -- start start 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424103ca0 0x7f7424074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7424104640 0x7f7424075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f742410e530 con 0x7f7424102aa0 2024-04-25T13:25:58.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f742410e3b0 con 0x7f7424104640 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f742b93f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f742410e6b0 con 0x7f7424103ca0 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49582/0 (socket says 172.21.15.37:49582) 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 -- 172.21.15.37:0/2290624608 learned_addr learned my addr 172.21.15.37:0/2290624608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f7428eda700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424103ca0 0x7f7424074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 -- 172.21.15.37:0/2290624608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424103ca0 msgr2=0x7f7424074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424103ca0 0x7f7424074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 -- 172.21.15.37:0/2290624608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7424104640 msgr2=0x7f7424075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7424104640 0x7f7424075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 -- 172.21.15.37:0/2290624608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7424075c20 con 0x7f7424102aa0 2024-04-25T13:25:58.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.440+0000 7f74296db700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7420009f50 tx=0x7f74200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.441+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420027070 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.441+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7420022470 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.441+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241a2d00 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.441+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742000f6c0 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.441+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241a31b0 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.442+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7424069730 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.443+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f742002a070 con 0x7f7424102aa0 2024-04-25T13:25:58.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.443+0000 7f741a7fc700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 0x7f74100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.444+0000 7f7428eda700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 0x7f74100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.444+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f74200d13c0 con 0x7f7424102aa0 2024-04-25T13:25:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.444+0000 7f7428eda700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 0x7f74100a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f741400ac50 tx=0x7f741400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:58.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.448+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7420013070 con 0x7f7424102aa0 2024-04-25T13:25:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.657+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7424063650 con 0x7f7424102aa0 2024-04-25T13:25:58.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.657+0000 7f741a7fc700 1 -- 172.21.15.37:0/2290624608 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5306 (secure 0 0 0) 0x7f742009ddb0 con 0x7f7424102aa0 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 91 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:fsid d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T13:16:08.851582+0000 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-25T13:25:07.524019+0000 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-04-25T13:25:58.658 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/3385916087,v1:172.21.15.37:6803/3385916087] [v2:172.21.15.37:6804/3385916087,v1:172.21.15.37:6805/3385916087] exists,up f78d67e4-010a-4965-86af-acd038f6cdf1 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/1009231193,v1:172.21.15.44:6801/1009231193] [v2:172.21.15.44:6802/1009231193,v1:172.21.15.44:6803/1009231193] exists,up 037053d7-433f-4d1a-9e7a-d44599f4a488 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6800/3261521434,v1:172.21.15.69:6801/3261521434] [v2:172.21.15.69:6802/3261521434,v1:172.21.15.69:6803/3261521434] exists,up 3317a25e-0c74-403e-94a3-fba65b8e6733 2024-04-25T13:25:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 27 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/1255089911,v1:172.21.15.37:6811/1255089911] [v2:172.21.15.37:6812/1255089911,v1:172.21.15.37:6813/1255089911] exists,up 97436b7d-2b40-4fbb-b523-4773cbf165f1 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 35 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/844548522,v1:172.21.15.44:6809/844548522] [v2:172.21.15.44:6810/844548522,v1:172.21.15.44:6811/844548522] exists,up 748fab07-a621-4314-a001-a402dc9d7d08 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 40 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6808/3958240806,v1:172.21.15.69:6809/3958240806] [v2:172.21.15.69:6810/3958240806,v1:172.21.15.69:6811/3958240806] exists,up 1dbcf0a2-1319-4cc8-9c58-d1d2a5520476 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 48 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/4082263223,v1:172.21.15.37:6819/4082263223] [v2:172.21.15.37:6820/4082263223,v1:172.21.15.37:6821/4082263223] exists,up d7fef8d2-092c-445a-aa3e-64878cfcd1a3 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 54 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6816/3060345546,v1:172.21.15.44:6817/3060345546] [v2:172.21.15.44:6818/3060345546,v1:172.21.15.44:6819/3060345546] exists,up d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 63 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6816/1610697891,v1:172.21.15.69:6817/1610697891] [v2:172.21.15.69:6818/1610697891,v1:172.21.15.69:6819/1610697891] exists,up 6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 71 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/682284085,v1:172.21.15.37:6827/682284085] [v2:172.21.15.37:6828/682284085,v1:172.21.15.37:6829/682284085] exists,up 3c7a5005-401a-4bcb-b6fc-357793f24756 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 76 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/2564518662,v1:172.21.15.44:6825/2564518662] [v2:172.21.15.44:6826/2564518662,v1:172.21.15.44:6827/2564518662] exists,up af8e1072-54de-4e47-9ca4-7b9fdcae474a 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 84 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6824/2045769419,v1:172.21.15.69:6825/2045769419] [v2:172.21.15.69:6826/2045769419,v1:172.21.15.69:6827/2045769419] exists,up 359839d6-6c86-46ea-be27-6f5d8fbcdece 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.0 [8,4] 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.6 [8,11] 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.8 [2,11] 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.14 [2,4] 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.7 [3,9] 2024-04-25T13:25:58.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.e [5,9] 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.1b [4,6] 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.25 [5,9] 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.27 [4,8] 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/3707854772 expires 2024-04-26T13:17:05.642821+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/3707854772 expires 2024-04-26T13:17:05.642821+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3175650939 expires 2024-04-26T13:17:05.642821+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1473000576 expires 2024-04-26T13:16:37.834096+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2107975660 expires 2024-04-26T13:16:37.834096+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/755315013 expires 2024-04-26T13:17:05.642821+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1321688497 expires 2024-04-26T13:17:05.642821+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/4026748437 expires 2024-04-26T13:16:37.834096+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2456399257 expires 2024-04-26T13:16:37.834096+0000 2024-04-25T13:25:58.661 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2107975660 expires 2024-04-26T13:16:37.834096+0000 2024-04-25T13:25:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.659+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 msgr2=0x7f74100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.659+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 0x7f74100a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f741400ac50 tx=0x7f741400a380 comp rx=0 tx=0).stop 2024-04-25T13:25:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.659+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 msgr2=0x7f74240767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.659+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7420009f50 tx=0x7f74200047b0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 shutdown_connections 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f74100a3c40 0x7f74100a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7424104640 0x7f7424075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102aa0 0x7f74240767e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 --2- 172.21.15.37:0/2290624608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7424103ca0 0x7f7424074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 >> 172.21.15.37:0/2290624608 conn(0x7f74240fe040 msgr2=0x7f7424107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 shutdown_connections 2024-04-25T13:25:58.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:58.660+0000 7f742b93f700 1 -- 172.21.15.37:0/2290624608 wait complete. 2024-04-25T13:25:59.041 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:58 smithi037 ceph-mon[29204]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:59.041 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:58 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2290624608' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:25:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:58 smithi069 ceph-mon[32534]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:58 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2290624608' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:25:59.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:58 smithi044 ceph-mon[32590]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:25:59.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:58 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2290624608' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T13:25:59.475 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T13:25:59.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:25:59.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 -- 172.21.15.37:0/4193531225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4102af0 msgr2=0x7f4cb4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/4193531225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4102af0 0x7f4cb4104ee0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009900 tx=0x7f4cac02a410 comp rx=0 tx=0).stop 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 -- 172.21.15.37:0/4193531225 shutdown_connections 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/4193531225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb4107d50 0x7f4cb410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/4193531225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4105420 0x7f4cb4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/4193531225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4102af0 0x7f4cb4104ee0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 -- 172.21.15.37:0/4193531225 >> 172.21.15.37:0/4193531225 conn(0x7f4cb40fc460 msgr2=0x7f4cb40fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 -- 172.21.15.37:0/4193531225 shutdown_connections 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.597+0000 7f4cb93c4640 1 -- 172.21.15.37:0/4193531225 wait complete. 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 Processor -- start 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 -- start start 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4102af0 0x7f4cb4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb4107d50 0x7f4cb419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410db60 con 0x7f4cb4105420 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410d9e0 con 0x7f4cb4107d50 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb93c4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb410dce0 con 0x7f4cb4102af0 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49602/0 (socket says 172.21.15.37:49602) 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb27fc640 1 -- 172.21.15.37:0/240131899 learned_addr learned my addr 172.21.15.37:0/240131899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb2ffd640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4102af0 0x7f4cb4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb27fc640 1 -- 172.21.15.37:0/240131899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4102af0 msgr2=0x7f4cb4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.598+0000 7f4cb27fc640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4102af0 0x7f4cb4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb27fc640 1 -- 172.21.15.37:0/240131899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb4107d50 msgr2=0x7f4cb419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb27fc640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb4107d50 0x7f4cb419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb27fc640 1 -- 172.21.15.37:0/240131899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb419feb0 con 0x7f4cb4105420 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb27fc640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b450 tx=0x7f4ca800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca8015020 con 0x7f4cb4105420 2024-04-25T13:25:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca8011d40 con 0x7f4cb4105420 2024-04-25T13:25:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41a01a0 con 0x7f4cb4105420 2024-04-25T13:25:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800f340 con 0x7f4cb4105420 2024-04-25T13:25:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.599+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb41a0720 con 0x7f4cb4105420 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.600+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb40fdf50 con 0x7f4cb4105420 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.601+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ca8018020 con 0x7f4cb4105420 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.602+0000 7f4c9bfff640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 0x7f4c9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.602+0000 7f4cb2ffd640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 0x7f4c9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.602+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f4ca80d0580 con 0x7f4cb4105420 2024-04-25T13:25:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.602+0000 7f4cb2ffd640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 0x7f4c9c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cac000c00 tx=0x7f4cac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:59.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.605+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ca801f020 con 0x7f4cb4105420 2024-04-25T13:25:59.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.793+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cb402d810 con 0x7f4cb4105420 2024-04-25T13:25:59.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.795+0000 7f4c9bfff640 1 -- 172.21.15.37:0/240131899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7f4ca800d040 con 0x7f4cb4105420 2024-04-25T13:25:59.795 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:25:59.796 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:25:59.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.796+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 msgr2=0x7f4c9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.796+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 0x7f4c9c0a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4cac000c00 tx=0x7f4cac0314a0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.796+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 msgr2=0x7f4cb41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.796+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b450 tx=0x7f4ca800b910 comp rx=0 tx=0).stop 2024-04-25T13:25:59.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 shutdown_connections 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cb4107d50 0x7f4cb419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4c9c0a3bb0 0x7f4c9c0a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cb4105420 0x7f4cb41987d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 --2- 172.21.15.37:0/240131899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4cb4102af0 0x7f4cb4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 >> 172.21.15.37:0/240131899 conn(0x7f4cb40fc460 msgr2=0x7f4cb4103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 shutdown_connections 2024-04-25T13:25:59.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.797+0000 7f4cb93c4640 1 -- 172.21.15.37:0/240131899 wait complete. 2024-04-25T13:25:59.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:25:59.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:25:59.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 -- 172.21.15.37:0/1445881072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0107d40 msgr2=0x7efde010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 --2- 172.21.15.37:0/1445881072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0107d40 0x7efde010a130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009ca0 tx=0x7efdd802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 -- 172.21.15.37:0/1445881072 shutdown_connections 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 --2- 172.21.15.37:0/1445881072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0107d40 0x7efde010a130 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 --2- 172.21.15.37:0/1445881072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0105410 0x7efde0107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 --2- 172.21.15.37:0/1445881072 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 -- 172.21.15.37:0/1445881072 >> 172.21.15.37:0/1445881072 conn(0x7efde00fc430 msgr2=0x7efde00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 -- 172.21.15.37:0/1445881072 shutdown_connections 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.914+0000 7efde5544640 1 -- 172.21.15.37:0/1445881072 wait complete. 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 Processor -- start 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 -- start start 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0107d40 0x7efde019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efde010dbd0 con 0x7efde0105410 2024-04-25T13:25:59.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efde010da50 con 0x7efde0107d40 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efde5544640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efde010dd50 con 0x7efde0102ae0 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efdde7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efdde7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55538/0 (socket says 172.21.15.37:55538) 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.915+0000 7efdde7fc640 1 -- 172.21.15.37:0/3716495565 learned_addr learned my addr 172.21.15.37:0/3716495565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 -- 172.21.15.37:0/3716495565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 msgr2=0x7efde0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efddeffd640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efddf7fe640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0107d40 0x7efde019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 -- 172.21.15.37:0/3716495565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0107d40 msgr2=0x7efde019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0107d40 0x7efde019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 -- 172.21.15.37:0/3716495565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde019ff20 con 0x7efde0105410 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efddeffd640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdde7fc640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efdd400b450 tx=0x7efdd400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:59.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4015020 con 0x7efde0105410 2024-04-25T13:25:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdd4011d40 con 0x7efde0105410 2024-04-25T13:25:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde01a0210 con 0x7efde0105410 2024-04-25T13:25:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400f340 con 0x7efde0105410 2024-04-25T13:25:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.916+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01a0830 con 0x7efde0105410 2024-04-25T13:25:59.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.917+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00fdf20 con 0x7efde0105410 2024-04-25T13:25:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.918+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efdd4018020 con 0x7efde0105410 2024-04-25T13:25:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.919+0000 7efdc7fff640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:25:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.919+0000 7efddeffd640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:25:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.919+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7efdd40d0520 con 0x7efde0105410 2024-04-25T13:25:59.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.919+0000 7efddeffd640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 0x7efdc80a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:25:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:25:59.922+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdd401f020 con 0x7efde0105410 2024-04-25T13:26:00.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:25:59 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/240131899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.110+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efde002d810 con 0x7efde0105410 2024-04-25T13:26:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.110+0000 7efdc7fff640 1 -- 172.21.15.37:0/3716495565 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7efdd400d040 con 0x7efde0105410 2024-04-25T13:26:00.111 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.112+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 msgr2=0x7efdc80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.112+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 0x7efdc80a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).stop 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.112+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 msgr2=0x7efde0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.112+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efdd400b450 tx=0x7efdd400b910 comp rx=0 tx=0).stop 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 shutdown_connections 2024-04-25T13:26:00.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde0107d40 0x7efde019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7efdc80a3c30 0x7efdc80a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efde0105410 0x7efde0198840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 --2- 172.21.15.37:0/3716495565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efde0102ae0 0x7efde0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 >> 172.21.15.37:0/3716495565 conn(0x7efde00fc430 msgr2=0x7efde0103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 shutdown_connections 2024-04-25T13:26:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.113+0000 7efde5544640 1 -- 172.21.15.37:0/3716495565 wait complete. 2024-04-25T13:26:00.124 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:26:00.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:25:59 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/240131899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:00.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:25:59 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/240131899' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:00.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 -- 172.21.15.37:0/1217208359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688102990 msgr2=0x7fd6881092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1217208359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688102990 0x7fd6881092b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ca0 tx=0x7fd68002a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 -- 172.21.15.37:0/1217208359 shutdown_connections 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1217208359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688102990 0x7fd6881092b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1217208359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688101fd0 0x7fd688102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1217208359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 0x7fd6881012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 -- 172.21.15.37:0/1217208359 >> 172.21.15.37:0/1217208359 conn(0x7fd6880fc3d0 msgr2=0x7fd6880fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 -- 172.21.15.37:0/1217208359 shutdown_connections 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 -- 172.21.15.37:0/1217208359 wait complete. 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.229+0000 7fd68fe72640 1 Processor -- start 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 -- start start 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 0x7fd68806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688102990 0x7fd68819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810c9e0 con 0x7fd688101fd0 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810c860 con 0x7fd688102990 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810cb60 con 0x7fd688100e80 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55562/0 (socket says 172.21.15.37:55562) 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 -- 172.21.15.37:0/1960289755 learned_addr learned my addr 172.21.15.37:0/1960289755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68dc0e640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 0x7fd68806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68e40f640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688102990 0x7fd68819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 -- 172.21.15.37:0/1960289755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 msgr2=0x7fd68806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 0x7fd68806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 -- 172.21.15.37:0/1960289755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688102990 msgr2=0x7fd68819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688102990 0x7fd68819f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 -- 172.21.15.37:0/1960289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819fd20 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68d40d640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd67800c970 tx=0x7fd67800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678007c10 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd678007db0 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.230+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819ffb0 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.231+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800f870 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.231+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a0470 con 0x7fd688101fd0 2024-04-25T13:26:00.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.231+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688108130 con 0x7fd688101fd0 2024-04-25T13:26:00.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.233+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd67800fa10 con 0x7fd688101fd0 2024-04-25T13:26:00.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.233+0000 7fd67effd640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 0x7fd6740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.233+0000 7fd68dc0e640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 0x7fd6740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.233+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd6780d0aa0 con 0x7fd688101fd0 2024-04-25T13:26:00.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.234+0000 7fd68dc0e640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 0x7fd6740a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd684005fd0 tx=0x7fd684005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.237+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd67809d580 con 0x7fd688101fd0 2024-04-25T13:26:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.423+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd68802d810 con 0x7fd688101fd0 2024-04-25T13:26:00.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.424+0000 7fd67effd640 1 -- 172.21.15.37:0/1960289755 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fd678017e80 con 0x7fd688101fd0 2024-04-25T13:26:00.424 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 msgr2=0x7fd6740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 0x7fd6740a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd684005fd0 tx=0x7fd684005850 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 msgr2=0x7fd68819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd67800c970 tx=0x7fd67800ce30 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 shutdown_connections 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd688102990 0x7fd68819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd6740a3bb0 0x7fd6740a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688101fd0 0x7fd68819cdf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 --2- 172.21.15.37:0/1960289755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd688100e80 0x7fd68806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 >> 172.21.15.37:0/1960289755 conn(0x7fd6880fc3d0 msgr2=0x7fd688105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 shutdown_connections 2024-04-25T13:26:00.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.426+0000 7fd68fe72640 1 -- 172.21.15.37:0/1960289755 wait complete. 2024-04-25T13:26:00.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:26:00.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 -- 172.21.15.37:0/2384242445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130103180 msgr2=0x7f21301055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 --2- 172.21.15.37:0/2384242445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130103180 0x7f21301055f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2124009900 tx=0x7f212402a410 comp rx=0 tx=0).stop 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 -- 172.21.15.37:0/2384242445 shutdown_connections 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 --2- 172.21.15.37:0/2384242445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f2130105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 --2- 172.21.15.37:0/2384242445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130103180 0x7f21301055f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 --2- 172.21.15.37:0/2384242445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2130100810 0x7f2130102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.541+0000 7f2137a10640 1 -- 172.21.15.37:0/2384242445 >> 172.21.15.37:0/2384242445 conn(0x7f21300fc430 msgr2=0x7f21300fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- 172.21.15.37:0/2384242445 shutdown_connections 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- 172.21.15.37:0/2384242445 wait complete. 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 Processor -- start 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- start start 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2130103180 0x7f213019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f21301a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114110 con 0x7f2130100810 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2130113f90 con 0x7f2130105b30 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2137a10640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114290 con 0x7f2130103180 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f21357ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f21357ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55574/0 (socket says 172.21.15.37:55574) 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f21357ac640 1 -- 172.21.15.37:0/3762342506 learned_addr learned my addr 172.21.15.37:0/3762342506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.542+0000 7f2135fad640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f21301a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 -- 172.21.15.37:0/3762342506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2130103180 msgr2=0x7f213019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2130103180 0x7f213019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 -- 172.21.15.37:0/3762342506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 msgr2=0x7f21301a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f21301a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 -- 172.21.15.37:0/3762342506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21301a6440 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f2135fad640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f21301a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f21357ac640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f212800e9f0 tx=0x7f212800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212800cd80 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21280054d0 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21301a6730 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128010690 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.543+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21301a6c30 con 0x7f2130100810 2024-04-25T13:26:00.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.544+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21300fdf20 con 0x7f2130100810 2024-04-25T13:26:00.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.545+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2128004e90 con 0x7f2130100810 2024-04-25T13:26:00.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.546+0000 7f212e7fc640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 0x7f21180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.546+0000 7f2134fab640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 0x7f21180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.546+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f21280d09d0 con 0x7f2130100810 2024-04-25T13:26:00.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.546+0000 7f2134fab640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 0x7f21180a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2124009230 tx=0x7f2124000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.550+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f212809d330 con 0x7f2130100810 2024-04-25T13:26:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.737+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2130062b50 con 0x7f2130100810 2024-04-25T13:26:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.738+0000 7f212e7fc640 1 -- 172.21.15.37:0/3762342506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f212809cc30 con 0x7f2130100810 2024-04-25T13:26:00.739 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:26:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 msgr2=0x7f21180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 0x7f21180a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2124009230 tx=0x7f2124000bc0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 msgr2=0x7f213019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f212800e9f0 tx=0x7f212800edf0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 shutdown_connections 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2130105b30 0x7f21301a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2130103180 0x7f213019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f21180a3bb0 0x7f21180a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 --2- 172.21.15.37:0/3762342506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2130100810 0x7f213019e820 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 >> 172.21.15.37:0/3762342506 conn(0x7f21300fc430 msgr2=0x7f2130101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 shutdown_connections 2024-04-25T13:26:00.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.740+0000 7f2137a10640 1 -- 172.21.15.37:0/3762342506 wait complete. 2024-04-25T13:26:00.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 -- 172.21.15.37:0/1480774551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103bc0 msgr2=0x7f805810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 --2- 172.21.15.37:0/1480774551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103bc0 0x7f805810a450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8048009ca0 tx=0x7f804802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 -- 172.21.15.37:0/1480774551 shutdown_connections 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 --2- 172.21.15.37:0/1480774551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103bc0 0x7f805810a450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 --2- 172.21.15.37:0/1480774551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058103200 0x7f8058103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 --2- 172.21.15.37:0/1480774551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058102010 0x7f8058102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 -- 172.21.15.37:0/1480774551 >> 172.21.15.37:0/1480774551 conn(0x7f80580fd580 msgr2=0x7f80580ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:00.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 -- 172.21.15.37:0/1480774551 shutdown_connections 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.853+0000 7f805d660640 1 -- 172.21.15.37:0/1480774551 wait complete. 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 Processor -- start 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 -- start start 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058102010 0x7f80581980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 0x7f805819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f805810dae0 con 0x7f8058103200 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f805810d960 con 0x7f8058103bc0 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f805d660640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f805810dc60 con 0x7f8058102010 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55596/0 (socket says 172.21.15.37:55596) 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 -- 172.21.15.37:0/1505344820 learned_addr learned my addr 172.21.15.37:0/1505344820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:00.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f8056ffd640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058102010 0x7f80581980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 -- 172.21.15.37:0/1505344820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058102010 msgr2=0x7f80581980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058102010 0x7f80581980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80567fc640 1 -- 172.21.15.37:0/1505344820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 msgr2=0x7f805819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.854+0000 7f80577fe640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 0x7f805819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f80567fc640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 0x7f805819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f80567fc640 1 -- 172.21.15.37:0/1505344820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805819fcd0 con 0x7f8058103200 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f80577fe640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 0x7f805819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f80567fc640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f804000d770 tx=0x7f804000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040017070 con 0x7f8058103200 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80400054d0 con 0x7f8058103200 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805819ffc0 con 0x7f8058103200 2024-04-25T13:26:00.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040004c30 con 0x7f8058103200 2024-04-25T13:26:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.855+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a0430 con 0x7f8058103200 2024-04-25T13:26:00.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.856+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80581092d0 con 0x7f8058103200 2024-04-25T13:26:00.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.857+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8040004e10 con 0x7f8058103200 2024-04-25T13:26:00.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.858+0000 7f804ffff640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:00.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.858+0000 7f8056ffd640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:00.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.858+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f80400cff60 con 0x7f8058103200 2024-04-25T13:26:00.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.858+0000 7f8056ffd640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 0x7f80440a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80500058c0 tx=0x7f8050005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:00.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:00.862+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f804009c920 con 0x7f8058103200 2024-04-25T13:26:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.088+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8058062bf0 con 0x7f8058103200 2024-04-25T13:26:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.088+0000 7f804ffff640 1 -- 172.21.15.37:0/1505344820 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7f804009c220 con 0x7f8058103200 2024-04-25T13:26:01.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:26:01.089 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:26:01.089 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 msgr2=0x7f80440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 0x7f80440a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f80500058c0 tx=0x7f8050005850 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 msgr2=0x7f80581985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f804000d770 tx=0x7f804000dc30 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 shutdown_connections 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8058103bc0 0x7f805819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f80440a3bb0 0x7f80440a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8058103200 0x7f80581985f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 --2- 172.21.15.37:0/1505344820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8058102010 0x7f80581980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 >> 172.21.15.37:0/1505344820 conn(0x7f80580fd580 msgr2=0x7f80581064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 shutdown_connections 2024-04-25T13:26:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.090+0000 7f805d660640 1 -- 172.21.15.37:0/1505344820 wait complete. 2024-04-25T13:26:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:00 smithi037 ceph-mon[29204]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3716495565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1960289755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:00 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3762342506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:26:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:00 smithi069 ceph-mon[32534]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:00 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3716495565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:00 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1960289755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:00 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3762342506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:00 smithi044 ceph-mon[32590]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:00 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3716495565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:00 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1960289755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:00 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3762342506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- 172.21.15.37:0/3798399156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44101fb0 msgr2=0x7eff441023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/3798399156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44101fb0 0x7eff441023d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009900 tx=0x7eff2c02a3f0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- 172.21.15.37:0/3798399156 shutdown_connections 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/3798399156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44103b60 0x7eff4410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/3798399156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/3798399156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44101fb0 0x7eff441023d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- 172.21.15.37:0/3798399156 >> 172.21.15.37:0/3798399156 conn(0x7eff440fd540 msgr2=0x7eff440ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- 172.21.15.37:0/3798399156 shutdown_connections 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- 172.21.15.37:0/3798399156 wait complete. 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 Processor -- start 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.204+0000 7eff4a2cf640 1 -- start start 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44101fb0 0x7eff44075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410db00 con 0x7eff44103b60 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410d980 con 0x7eff441031a0 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7eff4410dc80 con 0x7eff44101fb0 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55618/0 (socket says 172.21.15.37:55618) 2024-04-25T13:26:01.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 -- 172.21.15.37:0/1655139011 learned_addr learned my addr 172.21.15.37:0/1655139011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff43fff640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44101fb0 0x7eff44075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff437fe640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 -- 172.21.15.37:0/1655139011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44101fb0 msgr2=0x7eff44075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44101fb0 0x7eff44075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 -- 172.21.15.37:0/1655139011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 msgr2=0x7eff44074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 -- 172.21.15.37:0/1655139011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff44075110 con 0x7eff44103b60 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff437fe640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4886c640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00cce0 tx=0x7eff3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c00f450 con 0x7eff44103b60 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c00faa0 con 0x7eff44103b60 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff441a2290 con 0x7eff44103b60 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c018b70 con 0x7eff44103b60 2024-04-25T13:26:01.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.205+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff441a2790 con 0x7eff44103b60 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.207+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff441092d0 con 0x7eff44103b60 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.208+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7eff3c018d10 con 0x7eff44103b60 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.209+0000 7eff417fa640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 0x7eff300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.209+0000 7eff43fff640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 0x7eff300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.209+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7eff3c0d0990 con 0x7eff44103b60 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.209+0000 7eff43fff640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 0x7eff300a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.213+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff3c014030 con 0x7eff44103b60 2024-04-25T13:26:01.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.401+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff44062b50 con 0x7eff44103b60 2024-04-25T13:26:01.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.402+0000 7eff417fa640 1 -- 172.21.15.37:0/1655139011 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7eff3c09d350 con 0x7eff44103b60 2024-04-25T13:26:01.403 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:26:01.403 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 msgr2=0x7eff300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 0x7eff300a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006010 tx=0x7eff2c0314a0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 msgr2=0x7eff44074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00cce0 tx=0x7eff3c00a620 comp rx=0 tx=0).stop 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 shutdown_connections 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7eff300a3bb0 0x7eff300a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff44103b60 0x7eff44074880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff441031a0 0x7eff44074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 --2- 172.21.15.37:0/1655139011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff44101fb0 0x7eff44075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.404+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 >> 172.21.15.37:0/1655139011 conn(0x7eff440fd540 msgr2=0x7eff441064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.405+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 shutdown_connections 2024-04-25T13:26:01.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.405+0000 7eff4a2cf640 1 -- 172.21.15.37:0/1655139011 wait complete. 2024-04-25T13:26:01.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:26:01.417 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:26:01.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 -- 172.21.15.37:0/3084114199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130107c70 msgr2=0x7f813010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 --2- 172.21.15.37:0/3084114199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130107c70 0x7f813010a0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8128009ca0 tx=0x7f812802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 -- 172.21.15.37:0/3084114199 shutdown_connections 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 --2- 172.21.15.37:0/3084114199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130107c70 0x7f813010a0c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 --2- 172.21.15.37:0/3084114199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 --2- 172.21.15.37:0/3084114199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130102a10 0x7f8130104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 -- 172.21.15.37:0/3084114199 >> 172.21.15.37:0/3084114199 conn(0x7f81300fc420 msgr2=0x7f81300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 -- 172.21.15.37:0/3084114199 shutdown_connections 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.521+0000 7f8135f36640 1 -- 172.21.15.37:0/3084114199 wait complete. 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 Processor -- start 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 -- start start 2024-04-25T13:26:01.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130107c70 0x7f813019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f813010db40 con 0x7f8130102a10 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d9c0 con 0x7f8130105340 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f8135f36640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f813010dcc0 con 0x7f8130107c70 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55626/0 (socket says 172.21.15.37:55626) 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 -- 172.21.15.37:0/1476592557 learned_addr learned my addr 172.21.15.37:0/1476592557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812ffff640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130107c70 0x7f813019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812effd640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 -- 172.21.15.37:0/1476592557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130107c70 msgr2=0x7f813019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130107c70 0x7f813019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 -- 172.21.15.37:0/1476592557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 msgr2=0x7f8130198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 -- 172.21.15.37:0/1476592557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813019fe60 con 0x7f8130102a10 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812effd640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.522+0000 7f812f7fe640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f812400b4a0 tx=0x7f812400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.523+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124014070 con 0x7f8130102a10 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.523+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124010ac0 con 0x7f8130102a10 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.523+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81301a0150 con 0x7f8130102a10 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.523+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124017c30 con 0x7f8130102a10 2024-04-25T13:26:01.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.523+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a0770 con 0x7f8130102a10 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.524+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81300fdf10 con 0x7f8130102a10 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.525+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8124017070 con 0x7f8130102a10 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.525+0000 7f812cff9640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 0x7f81180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.526+0000 7f812effd640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 0x7f81180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.526+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f81240cfb40 con 0x7f8130102a10 2024-04-25T13:26:01.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.526+0000 7f812effd640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 0x7f81180a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8130199750 tx=0x7f8120005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.530+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f812409c500 con 0x7f8130102a10 2024-04-25T13:26:01.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.715+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8130062b50 con 0x7f8130102a10 2024-04-25T13:26:01.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.715+0000 7f812cff9640 1 -- 172.21.15.37:0/1476592557 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f812409be00 con 0x7f8130102a10 2024-04-25T13:26:01.716 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 msgr2=0x7f81180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 0x7f81180a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8130199750 tx=0x7f8120005d30 comp rx=0 tx=0).stop 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 msgr2=0x7f8130198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f812400b4a0 tx=0x7f812400b960 comp rx=0 tx=0).stop 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 shutdown_connections 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8130107c70 0x7f813019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8130105340 0x7f8130198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f81180a3c30 0x7f81180a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 --2- 172.21.15.37:0/1476592557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8130102a10 0x7f8130198240 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 >> 172.21.15.37:0/1476592557 conn(0x7f81300fc420 msgr2=0x7f8130103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 shutdown_connections 2024-04-25T13:26:01.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.717+0000 7f8135f36640 1 -- 172.21.15.37:0/1476592557 wait complete. 2024-04-25T13:26:01.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- 172.21.15.37:0/2824195254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 msgr2=0x7f20801012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 --2- 172.21.15.37:0/2824195254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f20801012a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2070009900 tx=0x7f207002a410 comp rx=0 tx=0).stop 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- 172.21.15.37:0/2824195254 shutdown_connections 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 --2- 172.21.15.37:0/2824195254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080102990 0x7f20801092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 --2- 172.21.15.37:0/2824195254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080101fd0 0x7f2080102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 --2- 172.21.15.37:0/2824195254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f20801012a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- 172.21.15.37:0/2824195254 >> 172.21.15.37:0/2824195254 conn(0x7f20800fc3d0 msgr2=0x7f20800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- 172.21.15.37:0/2824195254 shutdown_connections 2024-04-25T13:26:01.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- 172.21.15.37:0/2824195254 wait complete. 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 Processor -- start 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.834+0000 7f20868bb640 1 -- start start 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 0x7f208019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080102990 0x7f208019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f208010c9d0 con 0x7f2080100e80 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f208010c850 con 0x7f2080102990 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f20868bb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f208010cb50 con 0x7f2080101fd0 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55646/0 (socket says 172.21.15.37:55646) 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 -- 172.21.15.37:0/3903980138 learned_addr learned my addr 172.21.15.37:0/3903980138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f2084e58640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080102990 0x7f208019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207f7fe640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 0x7f208019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 -- 172.21.15.37:0/3903980138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 msgr2=0x7f208019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 0x7f208019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 -- 172.21.15.37:0/3903980138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080102990 msgr2=0x7f208019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080102990 0x7f208019f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 -- 172.21.15.37:0/3903980138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208019fdf0 con 0x7f2080100e80 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207f7fe640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 0x7f208019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:01.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.835+0000 7f207ffff640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2070005940 tx=0x7f2070004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.838+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207003f030 con 0x7f2080100e80 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.838+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2070003cb0 con 0x7f2080100e80 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.838+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a0020 con 0x7f2080100e80 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.838+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070035070 con 0x7f2080100e80 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.838+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a0560 con 0x7f2080100e80 2024-04-25T13:26:01.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.839+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2080108210 con 0x7f2080100e80 2024-04-25T13:26:01.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.840+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2070042030 con 0x7f2080100e80 2024-04-25T13:26:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.841+0000 7f207d7fa640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 0x7f206c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.841+0000 7f207f7fe640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 0x7f206c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.841+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f20700f0d00 con 0x7f2080100e80 2024-04-25T13:26:01.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.841+0000 7f207f7fe640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 0x7f206c0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f208006af20 tx=0x7f206800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:01.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:01.847+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2070003e60 con 0x7f2080100e80 2024-04-25T13:26:02.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.033+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2080062b80 con 0x7f2080100e80 2024-04-25T13:26:02.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.034+0000 7f207d7fa640 1 -- 172.21.15.37:0/3903980138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f20700bd6c0 con 0x7f2080100e80 2024-04-25T13:26:02.034 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:26:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 msgr2=0x7f206c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 0x7f206c0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f208006af20 tx=0x7f206800a560 comp rx=0 tx=0).stop 2024-04-25T13:26:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 msgr2=0x7f208006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2070005940 tx=0x7f2070004430 comp rx=0 tx=0).stop 2024-04-25T13:26:02.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 shutdown_connections 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2080102990 0x7f208019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2080101fd0 0x7f208019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f206c0a3c30 0x7f206c0a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 --2- 172.21.15.37:0/3903980138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2080100e80 0x7f208006b1b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 >> 172.21.15.37:0/3903980138 conn(0x7f20800fc3d0 msgr2=0x7f20801053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 shutdown_connections 2024-04-25T13:26:02.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.036+0000 7f20868bb640 1 -- 172.21.15.37:0/3903980138 wait complete. 2024-04-25T13:26:02.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:01 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1505344820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:01 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1655139011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:01 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1476592557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:02.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:26:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:01 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1505344820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:01 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1655139011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:01 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1476592557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:02.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:01 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1505344820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:01 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1655139011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:02.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:01 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1476592557' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 -- 172.21.15.37:0/2986665468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 msgr2=0x7f9a1c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/2986665468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c10a0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009ca0 tx=0x7f9a1402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 -- 172.21.15.37:0/2986665468 shutdown_connections 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/2986665468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c10a0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/2986665468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c1053a0 0x7f9a1c107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/2986665468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a1c102a70 0x7f9a1c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 -- 172.21.15.37:0/2986665468 >> 172.21.15.37:0/2986665468 conn(0x7f9a1c0fc420 msgr2=0x7f9a1c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 -- 172.21.15.37:0/2986665468 shutdown_connections 2024-04-25T13:26:02.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.151+0000 7f9a20c2c640 1 -- 172.21.15.37:0/2986665468 wait complete. 2024-04-25T13:26:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 Processor -- start 2024-04-25T13:26:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 -- start start 2024-04-25T13:26:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a1c102a70 0x7f9a1c1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c1053a0 0x7f9a1c100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c10db10 con 0x7f9a1c107cd0 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c10d990 con 0x7f9a1c1053a0 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a20c2c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c10dc90 con 0x7f9a1c102a70 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a1ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a1ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55666/0 (socket says 172.21.15.37:55666) 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.152+0000 7f9a1ad9d640 1 -- 172.21.15.37:0/1414000284 learned_addr learned my addr 172.21.15.37:0/1414000284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 -- 172.21.15.37:0/1414000284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a1c102a70 msgr2=0x7f9a1c1023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a1c102a70 0x7f9a1c1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 -- 172.21.15.37:0/1414000284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c1053a0 msgr2=0x7f9a1c100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c1053a0 0x7f9a1c100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 -- 172.21.15.37:0/1414000284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c1017f0 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a1ad9d640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a14033040 tx=0x7f9a14003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004040 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14004690 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c1a2240 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1402fe70 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.153+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a27e0 con 0x7f9a1c107cd0 2024-04-25T13:26:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.154+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c0fdf10 con 0x7f9a1c107cd0 2024-04-25T13:26:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.155+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a1402f610 con 0x7f9a1c107cd0 2024-04-25T13:26:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.156+0000 7f9a0b7fe640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 0x7f9a040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.156+0000 7f9a1a59c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 0x7f9a040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.156+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f9a140f1e30 con 0x7f9a1c107cd0 2024-04-25T13:26:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.156+0000 7f9a1a59c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 0x7f9a040a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00a8f0 tx=0x7f9a0c005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.160+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a140be790 con 0x7f9a1c107cd0 2024-04-25T13:26:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.345+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9a1c062b50 con 0x7f9a1c107cd0 2024-04-25T13:26:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.346+0000 7f9a0b7fe640 1 -- 172.21.15.37:0/1414000284 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f9a14039e10 con 0x7f9a1c107cd0 2024-04-25T13:26:02.346 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:26:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 msgr2=0x7f9a040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 0x7f9a040a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00a8f0 tx=0x7f9a0c005bf0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 msgr2=0x7f9a1c100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a14033040 tx=0x7f9a14003ba0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 shutdown_connections 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f9a040a3c30 0x7f9a040a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a1c107cd0 0x7f9a1c100f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a1c1053a0 0x7f9a1c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 --2- 172.21.15.37:0/1414000284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9a1c102a70 0x7f9a1c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 >> 172.21.15.37:0/1414000284 conn(0x7f9a1c0fc420 msgr2=0x7f9a1c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 shutdown_connections 2024-04-25T13:26:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.348+0000 7f9a20c2c640 1 -- 172.21.15.37:0/1414000284 wait complete. 2024-04-25T13:26:02.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.460+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3081062410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58100e60 msgr2=0x7fdd58101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.460+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3081062410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58100e60 0x7fdd58101280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009ca0 tx=0x7fdd4802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3081062410 shutdown_connections 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3081062410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102a00 0x7fdd58109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3081062410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58102040 0x7fdd581024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3081062410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58100e60 0x7fdd58101280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3081062410 >> 172.21.15.37:0/3081062410 conn(0x7fdd580fc420 msgr2=0x7fdd580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3081062410 shutdown_connections 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3081062410 wait complete. 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 Processor -- start 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.461+0000 7fdd5c9ad640 1 -- start start 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58100e60 0x7fdd5810b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102040 0x7fdd58109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810dad0 con 0x7fdd58102a00 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810d950 con 0x7fdd58102040 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810dc50 con 0x7fdd58100e60 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55686/0 (socket says 172.21.15.37:55686) 2024-04-25T13:26:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 -- 172.21.15.37:0/3076444394 learned_addr learned my addr 172.21.15.37:0/3076444394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5659c640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58100e60 0x7fdd5810b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd55d9b640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102040 0x7fdd58109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 -- 172.21.15.37:0/3076444394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58100e60 msgr2=0x7fdd5810b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58100e60 0x7fdd5810b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 -- 172.21.15.37:0/3076444394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102040 msgr2=0x7fdd58109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102040 0x7fdd58109d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 -- 172.21.15.37:0/3076444394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5810aaf0 con 0x7fdd58102a00 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd56d9d640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d750 tx=0x7fdd5000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50017070 con 0x7fdd58102a00 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd500097c0 con 0x7fdd58102a00 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581a2230 con 0x7fdd58102a00 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004ba0 con 0x7fdd58102a00 2024-04-25T13:26:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.462+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581a2730 con 0x7fdd58102a00 2024-04-25T13:26:02.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.463+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd58108200 con 0x7fdd58102a00 2024-04-25T13:26:02.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.465+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd50009c90 con 0x7fdd58102a00 2024-04-25T13:26:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.465+0000 7fdd477fe640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.466+0000 7fdd5659c640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.466+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fdd500cfda0 con 0x7fdd58102a00 2024-04-25T13:26:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.466+0000 7fdd5659c640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 0x7fdd400a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f80 tx=0x7fdd48005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.470+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd5009c760 con 0x7fdd58102a00 2024-04-25T13:26:02.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.694+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd58062b50 con 0x7fdd58102a00 2024-04-25T13:26:02.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.695+0000 7fdd477fe640 1 -- 172.21.15.37:0/3076444394 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7fdd5009c060 con 0x7fdd58102a00 2024-04-25T13:26:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:26:02.695 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:26:02.696 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 msgr2=0x7fdd400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 0x7fdd400a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f80 tx=0x7fdd48005dc0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 msgr2=0x7fdd5810a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000d750 tx=0x7fdd5000db50 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 shutdown_connections 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fdd400a3bb0 0x7fdd400a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58102a00 0x7fdd5810a260 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd58102040 0x7fdd58109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 --2- 172.21.15.37:0/3076444394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdd58100e60 0x7fdd5810b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 >> 172.21.15.37:0/3076444394 conn(0x7fdd580fc420 msgr2=0x7fdd581053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 shutdown_connections 2024-04-25T13:26:02.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.697+0000 7fdd5c9ad640 1 -- 172.21.15.37:0/3076444394 wait complete. 2024-04-25T13:26:02.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:26:02.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 -- 172.21.15.37:0/3683453326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c1031a0 msgr2=0x7f8a9c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/3683453326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c1031a0 0x7f8a9c103620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009900 tx=0x7f8a8802a410 comp rx=0 tx=0).stop 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 -- 172.21.15.37:0/3683453326 shutdown_connections 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/3683453326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a9c103b60 0x7f8a9c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/3683453326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c1031a0 0x7f8a9c103620 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/3683453326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 -- 172.21.15.37:0/3683453326 >> 172.21.15.37:0/3683453326 conn(0x7f8a9c0fd540 msgr2=0x7f8a9c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 -- 172.21.15.37:0/3683453326 shutdown_connections 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.810+0000 7f8aa2daf640 1 -- 172.21.15.37:0/3683453326 wait complete. 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 Processor -- start 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 -- start start 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a9c1031a0 0x7f8a9c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 0x7f8a9c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c10db50 con 0x7f8a9c101fb0 2024-04-25T13:26:02.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c10d9d0 con 0x7f8a9c1031a0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c10dcd0 con 0x7f8a9c103b60 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55702/0 (socket says 172.21.15.37:55702) 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 -- 172.21.15.37:0/343974213 learned_addr learned my addr 172.21.15.37:0/343974213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa134c640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 0x7f8a9c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 -- 172.21.15.37:0/343974213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 msgr2=0x7f8a9c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 0x7f8a9c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 -- 172.21.15.37:0/343974213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a9c1031a0 msgr2=0x7f8a9c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a9c1031a0 0x7f8a9c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 -- 172.21.15.37:0/343974213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c075110 con 0x7f8a9c101fb0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa134c640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 0x7f8a9c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa0b4b640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b450 tx=0x7f8a9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90015020 con 0x7f8a9c101fb0 2024-04-25T13:26:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.811+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c1a2390 con 0x7f8a9c101fb0 2024-04-25T13:26:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.812+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90011d40 con 0x7f8a9c101fb0 2024-04-25T13:26:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.812+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000f340 con 0x7f8a9c101fb0 2024-04-25T13:26:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.812+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c1a29b0 con 0x7f8a9c101fb0 2024-04-25T13:26:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.813+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c1093a0 con 0x7f8a9c101fb0 2024-04-25T13:26:02.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.814+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a90018020 con 0x7f8a9c101fb0 2024-04-25T13:26:02.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.814+0000 7f8a99ffb640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:02.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.815+0000 7f8a9bfff640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:02.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.815+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f8a900cffe0 con 0x7f8a9c101fb0 2024-04-25T13:26:02.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.815+0000 7f8a9bfff640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a880096c0 tx=0x7f8a88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:02.819+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a9001f020 con 0x7f8a9c101fb0 2024-04-25T13:26:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.007+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a9c062b50 con 0x7f8a9c101fb0 2024-04-25T13:26:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.008+0000 7f8a99ffb640 1 -- 172.21.15.37:0/343974213 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22605 (secure 0 0 0) 0x7f8a9000d040 con 0x7f8a9c101fb0 2024-04-25T13:26:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:26:03.010 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:25:07.524019+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.010+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 msgr2=0x7f8a8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.010+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8a880096c0 tx=0x7f8a88000bc0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.010+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 msgr2=0x7f8a9c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.010+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b450 tx=0x7f8a9000b910 comp rx=0 tx=0).stop 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 shutdown_connections 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a9c103b60 0x7f8a9c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a9c1031a0 0x7f8a9c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f8a8c0a3bb0 0x7f8a8c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 --2- 172.21.15.37:0/343974213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9c101fb0 0x7f8a9c075d00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 >> 172.21.15.37:0/343974213 conn(0x7f8a9c0fd540 msgr2=0x7f8a9c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 shutdown_connections 2024-04-25T13:26:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.011+0000 7f8aa2daf640 1 -- 172.21.15.37:0/343974213 wait complete. 2024-04-25T13:26:03.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:02 smithi037 ceph-mon[29204]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:03.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:02 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3903980138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:02 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1414000284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:02 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3076444394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:03.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T13:24:11.206089+0000', 'modified': '2024-04-25T13:24:15.243835+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.69:6839/1356630935', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.69:6838', 'nonce': 1356630935}, {'type': 'v1', 'addr': '172.21.15.69:6839', 'nonce': 1356630935}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24527': {'gid': 24527, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.44:6839/1296390680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 1296390680}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 1296390680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24527, 'qdb_cluster': [24527]} 2024-04-25T13:26:03.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:26:03.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 -- 172.21.15.37:0/2306950440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614105400 msgr2=0x7fd6141077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 --2- 172.21.15.37:0/2306950440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614105400 0x7fd6141077f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd608009900 tx=0x7fd60802a410 comp rx=0 tx=0).stop 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 -- 172.21.15.37:0/2306950440 shutdown_connections 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 --2- 172.21.15.37:0/2306950440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614107d30 0x7fd61410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 --2- 172.21.15.37:0/2306950440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614105400 0x7fd6141077f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 --2- 172.21.15.37:0/2306950440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614102ad0 0x7fd614104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.125+0000 7fd61b39b640 1 -- 172.21.15.37:0/2306950440 >> 172.21.15.37:0/2306950440 conn(0x7fd6140fc4c0 msgr2=0x7fd6140fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- 172.21.15.37:0/2306950440 shutdown_connections 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- 172.21.15.37:0/2306950440 wait complete. 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 Processor -- start 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- start start 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614102ad0 0x7fd614198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 0x7fd6141987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410db90 con 0x7fd614107d30 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410da10 con 0x7fd614102ad0 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd61b39b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd61410dd10 con 0x7fd614105400 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd619938640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd619938640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55732/0 (socket says 172.21.15.37:55732) 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.126+0000 7fd619938640 1 -- 172.21.15.37:0/4135407460 learned_addr learned my addr 172.21.15.37:0/4135407460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd618936640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 0x7fd6141987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619137640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614102ad0 0x7fd614198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 -- 172.21.15.37:0/4135407460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 msgr2=0x7fd6141987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 0x7fd6141987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 -- 172.21.15.37:0/4135407460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614102ad0 msgr2=0x7fd614198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614102ad0 0x7fd614198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 -- 172.21.15.37:0/4135407460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61419fea0 con 0x7fd614107d30 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd618936640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 0x7fd6141987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd619938640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d750 tx=0x7fd60c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd6127fc640 1 -- 172.21.15.37:0/4135407460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c017070 con 0x7fd614107d30 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141a0190 con 0x7fd614107d30 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd6127fc640 1 -- 172.21.15.37:0/4135407460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0054d0 con 0x7fd614107d30 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd6127fc640 1 -- 172.21.15.37:0/4135407460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004c60 con 0x7fd614107d30 2024-04-25T13:26:03.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.127+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a0690 con 0x7fd614107d30 2024-04-25T13:26:03.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:02 smithi069 ceph-mon[32534]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:03.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:02 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3903980138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:02 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1414000284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:02 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3076444394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:03.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.129+0000 7fd6127fc640 1 -- 172.21.15.37:0/4135407460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd60c009bd0 con 0x7fd614107d30 2024-04-25T13:26:03.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 Processor -- start 2024-04-25T13:26:03.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 -- start start 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfc3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfc51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648ecf28c50 con 0x5648ecfc5710 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648ecec5830 con 0x5648ecfc4d50 2024-04-25T13:26:03.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd61e191740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648ecf24c10 con 0x5648ecfc3b80 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd6127fc640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 0x7fd5fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55748/0 (socket says 172.21.15.37:55748) 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 -- 172.21.15.37:0/891905778 learned_addr learned my addr 172.21.15.37:0/891905778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfc3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd619938640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 0x7fd5fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd619137640 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfc51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd6127fc640 1 -- 172.21.15.37:0/4135407460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd60c0cfec0 con 0x7fd614107d30 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 -- 172.21.15.37:0/891905778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 msgr2=0x5648ecfc3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfc3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 -- 172.21.15.37:0/891905778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 msgr2=0x5648ecfc51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfc51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.130+0000 7fd618936640 1 -- 172.21.15.37:0/891905778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ecc3b2d0 con 0x5648ecfc5710 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.131+0000 7fd618936640 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd608005c40 tx=0x7fd608000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb7e38a849b3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.131+0000 7fd619938640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 0x7fd5fc0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009610 tx=0x7fd60c066c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.131+0000 7fd5fb7fe640 1 -- 172.21.15.37:0/891905778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608035040 con 0x5648ecfc5710 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.131+0000 7fd5fb7fe640 1 -- 172.21.15.37:0/891905778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd608031d10 con 0x5648ecfc5710 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.131+0000 7fd5fb7fe640 1 -- 172.21.15.37:0/891905778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608005410 con 0x5648ecfc5710 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- 172.21.15.37:0/891905778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 msgr2=0x5648ecfcbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd608005c40 tx=0x7fd608000bc0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- 172.21.15.37:0/891905778 shutdown_connections 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc5710 0x5648ecfcbfa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfc51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- 172.21.15.37:0/891905778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfc3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- 172.21.15.37:0/891905778 >> 172.21.15.37:0/891905778 conn(0x5648ecfbf070 msgr2=0x5648ecfbf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- 172.21.15.37:0/891905778 shutdown_connections 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- 172.21.15.37:0/891905778 wait complete. 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 -- start start 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 Processor -- start 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc3b80 0x5648ecfcdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.132+0000 7fd61e191740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfcc6c0 con 0x5648ecfc4d50 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648ecec3470 con 0x5648ecfc3b80 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfcc840 con 0x5648ecfc5710 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 10 client.0 _ll_register_callbacks cb 0x5648ecee8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55752/0 (socket says 172.21.15.37:55752) 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 -- 172.21.15.37:0/3743547115 learned_addr learned my addr 172.21.15.37:0/3743547115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd618936640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc3b80 0x5648ecfcdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd618936640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 -- 172.21.15.37:0/3743547115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 msgr2=0x5648ecfd5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 -- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc3b80 msgr2=0x5648ecfcdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc3b80 0x5648ecfcdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 -- 172.21.15.37:0/3743547115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ecd056b0 con 0x5648ecfc4d50 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd619137640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd604011460 tx=0x7fd604011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5de4ea354832d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T13:26:03.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604012710 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd6040128b0 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd608031d10 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd604012ab0 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 10 client.15180 fetch_fsmap learned FSMap version 20 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.133+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5648ecfd8940 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd608031d10 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd604025420 con 0x5648ecfc4d50 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 10 client.15180 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:26:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 20 client.15180 populate_metadata read hostname 'smithi037' 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ecfdafc0 0x5648ecfdc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 4 client.15180 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 --> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648ecf36670 con 0x5648ecfdafc0 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd618936640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ecfdafc0 0x5648ecfdc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd618936640 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ecfdafc0 0x5648ecfdc430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3d3c9529b517e5 server_cookie=54c85cd9096f3f4a in_seq=0 out_seq=0 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.134+0000 7fd5f8ff9640 10 client.15180 ms_handle_connect on v2:172.21.15.44:6838/1296390680 2024-04-25T13:26:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/3743547115 <== mds.0 v2:172.21.15.44:6838/1296390680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648ecf36670 con 0x5648ecfdafc0 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd5f8ff9640 10 client.15180 handle_command_reply: tid=0 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 client.15180 shutdown 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 msgr2=0x5648ecfce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd604011460 tx=0x7fd604011920 comp rx=0 tx=0).stop 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 shutdown_connections 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ecfdafc0 0x5648ecfdc430 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfce4c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd5f8ff9640 0 client.15180 ms_handle_reset on v2:172.21.15.44:6838/1296390680 2024-04-25T13:26:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 --2- 172.21.15.37:0/3743547115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc3b80 0x5648ecfcdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 >> 172.21.15.37:0/3743547115 conn(0x5648ecfbf070 msgr2=0x5648ecfbe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.135+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 shutdown_connections 2024-04-25T13:26:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.136+0000 7fd61e191740 1 -- 172.21.15.37:0/3743547115 wait complete. 2024-04-25T13:26:03.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.136+0000 7fd61e191740 20 client.15180 trim_cache size 0 max 16384 2024-04-25T13:26:03.148 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:02 smithi044 ceph-mon[32590]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:03.148 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:02 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3903980138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:02 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1414000284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:26:03.149 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:02 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3076444394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:26:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 Processor -- start 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 -- start start 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc5710 0x5648ecfcdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfda770 con 0x5648ecfc4d50 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfdae40 con 0x5648ecfc5710 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd61e191740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfdb510 con 0x5648ecfc3b80 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55768/0 (socket says 172.21.15.37:55768) 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 -- 172.21.15.37:0/4159717765 learned_addr learned my addr 172.21.15.37:0/4159717765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd618936640 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 -- 172.21.15.37:0/4159717765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 msgr2=0x5648ecfba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 -- 172.21.15.37:0/4159717765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc5710 msgr2=0x5648ecfcdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc5710 0x5648ecfcdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.154+0000 7fd619137640 1 -- 172.21.15.37:0/4159717765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ecfd6980 con 0x5648ecfc4d50 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd619137640 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd604005c50 tx=0x7fd604009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f667744faa705e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd5faffd640 1 -- 172.21.15.37:0/4159717765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604012e50 con 0x5648ecfc4d50 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd5faffd640 1 -- 172.21.15.37:0/4159717765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd604025420 con 0x5648ecfc4d50 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd5faffd640 1 -- 172.21.15.37:0/4159717765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604003c60 con 0x5648ecfc4d50 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 -- 172.21.15.37:0/4159717765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 msgr2=0x5648ecfd54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd604005c50 tx=0x7fd604009c90 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 -- 172.21.15.37:0/4159717765 shutdown_connections 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc5710 0x5648ecfcdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc4d50 0x5648ecfd54c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 --2- 172.21.15.37:0/4159717765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc3b80 0x5648ecfba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 -- 172.21.15.37:0/4159717765 >> 172.21.15.37:0/4159717765 conn(0x5648ecfbf070 msgr2=0x5648ecfbf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 -- 172.21.15.37:0/4159717765 shutdown_connections 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.155+0000 7fd61e191740 1 -- 172.21.15.37:0/4159717765 wait complete. 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 -- start start 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 Processor -- start 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfd1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfdc2b0 con 0x5648ecfc3b80 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfdbbe0 con 0x5648ecfc4d50 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648ecfdc980 con 0x5648ecfc5710 2024-04-25T13:26:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd61e191740 10 client.0 _ll_register_callbacks cb 0x7fd6040095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55770/0 (socket says 172.21.15.37:55770) 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 -- 172.21.15.37:0/1137299993 learned_addr learned my addr 172.21.15.37:0/1137299993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 -- 172.21.15.37:0/1137299993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 msgr2=0x5648ecfd4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 -- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 msgr2=0x5648ecfd1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfd1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.156+0000 7fd618936640 1 -- 172.21.15.37:0/1137299993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ecfd4a50 con 0x5648ecfc3b80 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd618936640 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd608005940 tx=0x7fd608032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecb0d146e670c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608035040 con 0x5648ecfc3b80 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7fd608005460 con 0x5648ecfc3b80 2024-04-25T13:26:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd604025420 con 0x5648ecfc3b80 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd608061050 con 0x5648ecfc3b80 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 10 client.15186 fetch_fsmap learned FSMap version 20 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5648ed0412c0 con 0x5648ecfc3b80 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd604025420 con 0x5648ecfc3b80 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd60803a040 con 0x5648ecfc3b80 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 10 client.15186 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 20 client.15186 populate_metadata read hostname 'smithi037' 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ed045a10 0x5648ed044720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 4 client.15186 mds_command: new command op to 24527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.157+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 --> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] -- command(tid 0: {"prefix": "status"}) -- 0x5648ecf304e0 con 0x5648ed045a10 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd619137640 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ed045a10 0x5648ed044720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd5f8ff9640 10 client.15186 ms_handle_connect on v2:172.21.15.44:6838/1296390680 2024-04-25T13:26:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd619137640 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ed045a10 0x5648ed044720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9328ba5b61251371 server_cookie=7cd6c835fe8b49e8 in_seq=0 out_seq=0 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd5f8ff9640 1 -- 172.21.15.37:0/1137299993 <== mds.0 v2:172.21.15.44:6838/1296390680 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5648ecf304e0 con 0x5648ed045a10 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd5f8ff9640 10 client.15186 handle_command_reply: tid=0 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.158+0000 7fd61e191740 1 client.15186 shutdown 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 msgr2=0x5648ecfd0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd608005940 tx=0x7fd608032d20 comp rx=0 tx=0).stop 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 shutdown_connections 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:6838/1296390680,v1:172.21.15.44:6839/1296390680] conn(0x5648ed045a10 0x5648ed044720 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648ecfc5710 0x5648ecfd4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd5f8ff9640 0 client.15186 ms_handle_reset on v2:172.21.15.44:6838/1296390680 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648ecfc4d50 0x5648ecfd1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 --2- 172.21.15.37:0/1137299993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648ecfc3b80 0x5648ecfd0f00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 >> 172.21.15.37:0/1137299993 conn(0x5648ecfbf070 msgr2=0x5648ecfba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 shutdown_connections 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 1 -- 172.21.15.37:0/1137299993 wait complete. 2024-04-25T13:26:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61e191740 20 client.15186 trim_cache size 0 max 16384 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "d766aeb0-0305-11ef-bc93-c7b262605968", 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24527, 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 111.290238366, 2024-04-25T13:26:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 20, 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 116.50931413000001 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 msgr2=0x7fd5fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 0x7fd5fc0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009610 tx=0x7fd60c066c10 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 msgr2=0x7fd61419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00d750 tx=0x7fd60c00dc10 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 shutdown_connections 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7fd5fc0a3bb0 0x7fd5fc0a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd614107d30 0x7fd61419f830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd614105400 0x7fd6141987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 --2- 172.21.15.37:0/4135407460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd614102ad0 0x7fd614198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 >> 172.21.15.37:0/4135407460 conn(0x7fd6140fc4c0 msgr2=0x7fd614104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.159+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 shutdown_connections 2024-04-25T13:26:03.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.160+0000 7fd61b39b640 1 -- 172.21.15.37:0/4135407460 wait complete. 2024-04-25T13:26:03.171 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:26:03.181 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:26:03.181 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:26:03.181 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:26:03.182 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T13:26:03.182 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:26:03.183 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:26:03.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T13:26:03.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 -- 172.21.15.37:0/206431034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 msgr2=0x7f4df410a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 --2- 172.21.15.37:0/206431034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df410a420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009ca0 tx=0x7f4dec02a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 -- 172.21.15.37:0/206431034 shutdown_connections 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 --2- 172.21.15.37:0/206431034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df410a420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 --2- 172.21.15.37:0/206431034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df41031d0 0x7f4df4103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 --2- 172.21.15.37:0/206431034 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df4101fe0 0x7f4df4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 -- 172.21.15.37:0/206431034 >> 172.21.15.37:0/206431034 conn(0x7f4df40fd570 msgr2=0x7f4df40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 -- 172.21.15.37:0/206431034 shutdown_connections 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.284+0000 7f4dfb819640 1 -- 172.21.15.37:0/206431034 wait complete. 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 Processor -- start 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 -- start start 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 0x7f4df4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df41031d0 0x7f4df4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410dc00 con 0x7f4df4103b90 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410da80 con 0x7f4df4101fe0 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4dfb819640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4df410dd80 con 0x7f4df41031d0 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55786/0 (socket says 172.21.15.37:55786) 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 -- 172.21.15.37:0/1662889993 learned_addr learned my addr 172.21.15.37:0/1662889993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df8db4640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df41031d0 0x7f4df4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df95b5640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 0x7f4df4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 -- 172.21.15.37:0/1662889993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df41031d0 msgr2=0x7f4df4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df41031d0 0x7f4df4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 -- 172.21.15.37:0/1662889993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 msgr2=0x7f4df4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 0x7f4df4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 -- 172.21.15.37:0/1662889993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df4075110 con 0x7f4df4103b90 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df95b5640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 0x7f4df4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.285+0000 7f4df9db6640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dec031a20 tx=0x7f4dec003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.286+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec004240 con 0x7f4df4103b90 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.286+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dec003e10 con 0x7f4df4103b90 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.286+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df41a2450 con 0x7f4df4103b90 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.286+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec02fd90 con 0x7f4df4103b90 2024-04-25T13:26:03.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.286+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41a2a70 con 0x7f4df4103b90 2024-04-25T13:26:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.287+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df4109370 con 0x7f4df4103b90 2024-04-25T13:26:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.288+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4dec004b40 con 0x7f4df4103b90 2024-04-25T13:26:03.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.288+0000 7f4dea7fc640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 0x7f4de00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.288+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f4dec0f1e00 con 0x7f4df4103b90 2024-04-25T13:26:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.288+0000 7f4df95b5640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 0x7f4de00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.289+0000 7f4df95b5640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 0x7f4de00a60e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4df4103030 tx=0x7f4df0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.292+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dec0be7c0 con 0x7f4df4103b90 2024-04-25T13:26:03.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.550+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f4df402d5c0 con 0x7f4df4103b90 2024-04-25T13:26:03.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.847+0000 7f4dea7fc640 1 -- 172.21.15.37:0/1662889993 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f4dec0be0c0 con 0x7f4df4103b90 2024-04-25T13:26:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.850+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 msgr2=0x7f4de00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.850+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 0x7f4de00a60e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4df4103030 tx=0x7f4df0008040 comp rx=0 tx=0).stop 2024-04-25T13:26:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.850+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 msgr2=0x7f4df4074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.850+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dec031a20 tx=0x7f4dec003c20 comp rx=0 tx=0).stop 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 shutdown_connections 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f4de00a3c20 0x7f4de00a60e0 secure :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4df4103030 tx=0x7f4df0008040 comp rx=0 tx=0).stop 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4df4103b90 0x7f4df4074880 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4df41031d0 0x7f4df4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 --2- 172.21.15.37:0/1662889993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4df4101fe0 0x7f4df4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 >> 172.21.15.37:0/1662889993 conn(0x7f4df40fd570 msgr2=0x7f4df4106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 shutdown_connections 2024-04-25T13:26:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.852+0000 7f4dfb819640 1 -- 172.21.15.37:0/1662889993 wait complete. 2024-04-25T13:26:03.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.980+0000 7f01c4e8a640 1 -- 172.21.15.37:0/2600232480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 msgr2=0x7f01c0103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.980+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/2600232480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0103630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009900 tx=0x7f01b402a410 comp rx=0 tx=0).stop 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 -- 172.21.15.37:0/2600232480 shutdown_connections 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/2600232480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0103b70 0x7f01c010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/2600232480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0103630 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/2600232480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0101fc0 0x7f01c01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 -- 172.21.15.37:0/2600232480 >> 172.21.15.37:0/2600232480 conn(0x7f01c00fd570 msgr2=0x7f01c00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 -- 172.21.15.37:0/2600232480 shutdown_connections 2024-04-25T13:26:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 -- 172.21.15.37:0/2600232480 wait complete. 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 Processor -- start 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 -- start start 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0101fc0 0x7f01c0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.981+0000 7f01c4e8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0103b70 0x7f01c0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01c010db50 con 0x7f01c01031b0 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01c010d9d0 con 0x7f01c0101fc0 2024-04-25T13:26:03.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f01c010dcd0 con 0x7f01c0103b70 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55832/0 (socket says 172.21.15.37:55832) 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 -- 172.21.15.37:0/3265746567 learned_addr learned my addr 172.21.15.37:0/3265746567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01be59c640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0101fc0 0x7f01c0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bed9d640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0103b70 0x7f01c0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0103b70 msgr2=0x7f01c0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0103b70 0x7f01c0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0101fc0 msgr2=0x7f01c0075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0101fc0 0x7f01c0075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 -- 172.21.15.37:0/3265746567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0075110 con 0x7f01c01031b0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01bdd9b640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005ab0 tx=0x7f01b4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b403f070 con 0x7f01c01031b0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c01a2270 con 0x7f01c01031b0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b4031c50 con 0x7f01c01031b0 2024-04-25T13:26:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035030 con 0x7f01c01031b0 2024-04-25T13:26:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.982+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a27c0 con 0x7f01c01031b0 2024-04-25T13:26:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.983+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c0109400 con 0x7f01c01031b0 2024-04-25T13:26:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.984+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f01b4042070 con 0x7f01c01031b0 2024-04-25T13:26:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.985+0000 7f01af7fe640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 0x7f01a80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.985+0000 7f01be59c640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 msgr2=0x7f01a80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.985+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f01b40f0bc0 con 0x7f01c01031b0 2024-04-25T13:26:03.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.985+0000 7f01be59c640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 0x7f01a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:26:03.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:03.990+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01b40bcf30 con 0x7f01c01031b0 2024-04-25T13:26:04.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:03 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/343974213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:04.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:03 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:26:04.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setuser ceph since I am not root 2024-04-25T13:26:04.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: ignoring --setgroup ceph since I am not root 2024-04-25T13:26:04.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:04.039+0000 7f5319605200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:26:04.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:03 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/343974213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:04.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:03 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:26:04.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: ignoring --setuser ceph since I am not root 2024-04-25T13:26:04.136 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: ignoring --setgroup ceph since I am not root 2024-04-25T13:26:04.136 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:03 smithi069 ceph-mgr[34410]: -- 172.21.15.69:0/4229240199 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db5f2c45a0 con 0x55db5f298800 2024-04-25T13:26:04.136 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:04.033+0000 7f84c66fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:26:04.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:03 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/343974213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:26:04.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:03 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T13:26:04.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: ignoring --setuser ceph since I am not root 2024-04-25T13:26:04.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: ignoring --setgroup ceph since I am not root 2024-04-25T13:26:04.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:03 smithi044 ceph-mgr[33990]: -- 172.21.15.44:0/3236225375 <== mon.2 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555d377c5a0 con 0x5555d3750c00 2024-04-25T13:26:04.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:04.037+0000 7f874eb7d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:26:04.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.180+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f01c002d5c0 con 0x7f01c01031b0 2024-04-25T13:26:04.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.184+0000 7f01af7fe640 1 -- 172.21.15.37:0/3265746567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v34)=0 v34) ==== 138+0+0 (secure 0 0 0) 0x7f01b40c1230 con 0x7f01c01031b0 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01be59c640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 msgr2=0x7f01a80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01be59c640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 0x7f01a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 msgr2=0x7f01a80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 0x7f01a80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 msgr2=0x7f01c0074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f01b4005ab0 tx=0x7f01b4003b90 comp rx=0 tx=0).stop 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 shutdown_connections 2024-04-25T13:26:04.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01c0103b70 0x7f01c0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f01a80a3bf0 0x7f01a80a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c01031b0 0x7f01c0074340 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 --2- 172.21.15.37:0/3265746567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01c0101fc0 0x7f01c0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 >> 172.21.15.37:0/3265746567 conn(0x7f01c00fd570 msgr2=0x7f01c01065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 shutdown_connections 2024-04-25T13:26:04.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.186+0000 7f01c4e8a640 1 -- 172.21.15.37:0/3265746567 wait complete. 2024-04-25T13:26:04.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 -- 172.21.15.37:0/869880444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55781053e0 msgr2=0x7f55781077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 --2- 172.21.15.37:0/869880444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55781053e0 0x7f55781077d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5564009900 tx=0x7f556402a410 comp rx=0 tx=0).stop 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 -- 172.21.15.37:0/869880444 shutdown_connections 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 --2- 172.21.15.37:0/869880444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5578107d10 0x7f557810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 --2- 172.21.15.37:0/869880444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55781053e0 0x7f55781077d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 --2- 172.21.15.37:0/869880444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578102ab0 0x7f5578104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 -- 172.21.15.37:0/869880444 >> 172.21.15.37:0/869880444 conn(0x7f55780fc420 msgr2=0x7f55780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 -- 172.21.15.37:0/869880444 shutdown_connections 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.305+0000 7f557f1f0640 1 -- 172.21.15.37:0/869880444 wait complete. 2024-04-25T13:26:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 Processor -- start 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 -- start start 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 0x7f5578198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578107d10 0x7f557819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f557810daf0 con 0x7f5578102ab0 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f557810d970 con 0x7f55781053e0 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557f1f0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f557810dc70 con 0x7f5578107d10 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55852/0 (socket says 172.21.15.37:55852) 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 -- 172.21.15.37:0/2870991013 learned_addr learned my addr 172.21.15.37:0/2870991013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557d78d640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578107d10 0x7f557819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f5577fff640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 0x7f5578198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 -- 172.21.15.37:0/2870991013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578107d10 msgr2=0x7f557819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:04.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578107d10 0x7f557819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 -- 172.21.15.37:0/2870991013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 msgr2=0x7f5578198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 0x7f5578198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f557cf8c640 1 -- 172.21.15.37:0/2870991013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557819fe40 con 0x7f5578102ab0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.306+0000 7f5577fff640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 0x7f5578198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f557cf8c640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00cd80 con 0x7f5578102ab0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c0108e0 con 0x7f5578102ab0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c018980 con 0x7f5578102ab0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781a0130 con 0x7f5578102ab0 2024-04-25T13:26:04.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.307+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a06a0 con 0x7f5578102ab0 2024-04-25T13:26:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.308+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55780fdf10 con 0x7f5578102ab0 2024-04-25T13:26:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.310+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f556c010430 con 0x7f5578102ab0 2024-04-25T13:26:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.311+0000 7f5575ffb640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 0x7f55680a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.311+0000 7f5577fff640 1 -- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 msgr2=0x7f55680a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:04.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.311+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f556c0d0cd0 con 0x7f5578102ab0 2024-04-25T13:26:04.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.311+0000 7f5577fff640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 0x7f55680a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:26:04.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.313+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f556c09d630 con 0x7f5578102ab0 2024-04-25T13:26:04.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.502+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f557802d5c0 con 0x7f5578102ab0 2024-04-25T13:26:04.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.505+0000 7f5575ffb640 1 -- 172.21.15.37:0/2870991013 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v35)=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7f556c09cf30 con 0x7f5578102ab0 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.507+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 msgr2=0x7f55680a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 0x7f55680a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 msgr2=0x7f5578198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f556c00e9f0 tx=0x7f556c00eeb0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 shutdown_connections 2024-04-25T13:26:04.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5578107d10 0x7f557819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55781053e0 0x7f5578198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f55680a3c70 0x7f55680a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 --2- 172.21.15.37:0/2870991013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102ab0 0x7f5578198220 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 >> 172.21.15.37:0/2870991013 conn(0x7f55780fc420 msgr2=0x7f5578104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 shutdown_connections 2024-04-25T13:26:04.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:04.508+0000 7f557f1f0640 1 -- 172.21.15.37:0/2870991013 wait complete. 2024-04-25T13:26:04.518 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:04.240+0000 7f5319605200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:04.518 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:04.352+0000 7f5319605200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:26:04.519 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T13:26:04.544 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T13:26:04.544 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T13:26:04.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:04.545 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T13:26:04.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:04.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T13:26:04.628 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-25T13:26:04.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:04.630 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T13:26:04.630 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:26:04.630 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:26:04.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T13:26:04.631 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:04.219+0000 7f84c66fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:04.631 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:04.320+0000 7f84c66fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:26:04.631 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:04.630+0000 7f84c66fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:26:04.639 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:04.228+0000 7f874eb7d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:04.639 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:04.334+0000 7f874eb7d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:26:04.772 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:04.680+0000 7f5319605200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:26:04.773 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:04.772+0000 7f5319605200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:26:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:26:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-mon[32534]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:26:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3265746567' entity='client.admin' 2024-04-25T13:26:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2870991013' entity='client.admin' 2024-04-25T13:26:04.886 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:04.716+0000 7f84c66fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:26:04.927 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:26:05.050 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:26:05.051 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-mon[29204]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:26:05.051 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3265746567' entity='client.admin' 2024-04-25T13:26:05.051 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:04 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2870991013' entity='client.admin' 2024-04-25T13:26:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1662889993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T13:26:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-mon[32590]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T13:26:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3265746567' entity='client.admin' 2024-04-25T13:26:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2870991013' entity='client.admin' 2024-04-25T13:26:05.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:04.654+0000 7f874eb7d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:26:05.140 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:04.742+0000 7f874eb7d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:26:06.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:06.054+0000 7f5319605200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:26:06.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:05.881+0000 7f84c66fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:26:06.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.051+0000 7f84c66fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:06.338 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:05.985+0000 7f874eb7d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:26:06.339 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:06.161+0000 7f874eb7d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:06.506 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.220+0000 7f84c66fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:26:06.570 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:06.234+0000 7f5319605200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:06.571 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:06.415+0000 7f5319605200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:26:06.797 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.506+0000 7f84c66fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:26:06.797 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.597+0000 7f84c66fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:26:06.797 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.797+0000 7f84c66fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:26:06.828 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:06.727+0000 7f5319605200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 -- 172.21.15.37:0/440659473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10108a80 msgr2=0x7f0a10108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/440659473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10108a80 0x7f0a10108e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009990 tx=0x7f0a04009ca0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 -- 172.21.15.37:0/440659473 shutdown_connections 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/440659473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10103450 0x7f0a1010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/440659473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10108a80 0x7f0a10108e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/440659473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10102a90 0x7f0a10102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 -- 172.21.15.37:0/440659473 >> 172.21.15.37:0/440659473 conn(0x7f0a10079bf0 msgr2=0x7f0a1007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 -- 172.21.15.37:0/440659473 shutdown_connections 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.837+0000 7f0a16bfb700 1 -- 172.21.15.37:0/440659473 wait complete. 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 Processor -- start 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 -- start start 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10102a90 0x7f0a10074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10108a80 0x7f0a100757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112c10 con 0x7f0a10103450 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112a90 con 0x7f0a10102a90 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a16bfb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112d90 con 0x7f0a10108a80 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55864/0 (socket says 172.21.15.37:55864) 2024-04-25T13:26:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 -- 172.21.15.37:0/588433387 learned_addr learned my addr 172.21.15.37:0/588433387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a15198700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10108a80 0x7f0a100757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a14997700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10102a90 0x7f0a10074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10108a80 msgr2=0x7f0a100757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10108a80 0x7f0a100757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10102a90 msgr2=0x7f0a10074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10102a90 0x7f0a10074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.838+0000 7f0a0ffff700 1 -- 172.21.15.37:0/588433387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a101a7670 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a0ffff700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00d980 tx=0x7f09fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc016070 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09fc004830 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a101a7960 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc005540 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.839+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a101a7ef0 con 0x7f0a10103450 2024-04-25T13:26:06.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.840+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a10069730 con 0x7f0a10103450 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.841+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f09fc004220 con 0x7f0a10103450 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.842+0000 7f0a0dffb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 0x7f0a000a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.842+0000 7f0a14997700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 msgr2=0x7f0a000a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.842+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f09fc0d01e0 con 0x7f0a10103450 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.842+0000 7f0a14997700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 0x7f0a000a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:26:06.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:06.846+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09fc01f020 con 0x7f0a10103450 2024-04-25T13:26:06.889 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:06.338+0000 7f874eb7d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:26:06.890 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:06.638+0000 7f874eb7d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:26:06.890 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:06.734+0000 7f874eb7d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:26:07.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.043+0000 7f0a14997700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 msgr2=0x7f0a000a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:07.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.043+0000 7f0a14997700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 0x7f0a000a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:26:07.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:06.828+0000 7f5319605200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:26:07.101 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:07.068+0000 7f5319605200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:26:07.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:06.990+0000 7f84c66fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:07.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.135+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a1002df80 con 0x7f0a10103450 2024-04-25T13:26:07.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.136+0000 7f0a0dffb700 1 -- 172.21.15.37:0/588433387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f09fc022350 con 0x7f0a10103450 2024-04-25T13:26:07.136 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:26:07.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 msgr2=0x7f0a000a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:26:07.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 0x7f0a000a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:07.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 msgr2=0x7f0a10075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:07.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00d980 tx=0x7f09fc00dc90 comp rx=0 tx=0).stop 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 shutdown_connections 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f0a000a3d10 0x7f0a000a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0a10102a90 0x7f0a10074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0a10108a80 0x7f0a100757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 --2- 172.21.15.37:0/588433387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a10103450 0x7f0a10075280 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 >> 172.21.15.37:0/588433387 conn(0x7f0a10079bf0 msgr2=0x7f0a10101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 shutdown_connections 2024-04-25T13:26:07.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:07.138+0000 7f0a16bfb700 1 -- 172.21.15.37:0/588433387 wait complete. 2024-04-25T13:26:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:07 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/588433387' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:26:07.390 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:06.943+0000 7f874eb7d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:26:07.390 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:07.143+0000 7f874eb7d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:07.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:07 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/588433387' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:26:07.517 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:07.277+0000 7f5319605200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:26:07.579 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:07 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/588433387' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T13:26:07.885 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:07.579+0000 7f84c66fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:26:07.922 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:26:07.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:07.923 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:26:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42081sec preferred_lft 42081sec 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:26:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T13:26:07.940 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-25T13:26:07.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:26:07.942 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T13:26:08.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:08.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:08.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:08.137 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:26:08.139 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:07.768+0000 7f874eb7d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:26:08.147 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:07.913+0000 7f5319605200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T13:26:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T13:26:08.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:08.163 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T13:26:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T13:26:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:26:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T13:26:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T13:26:08.237 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T13:26:08.238 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T13:26:08.286 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:07.996+0000 7f84c66fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:26:08.287 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:08.090+0000 7f84c66fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:26:08.287 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:08.287+0000 7f84c66fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:26:08.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:08.428 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:08.380+0000 7f5319605200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:26:08.515 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:08.205+0000 7f874eb7d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:26:08.515 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:08.305+0000 7f874eb7d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:26:08.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:08.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:08.530 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T13:26:08.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:08.545 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T13:26:08.602 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:26:08.602 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T13:26:08.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:26:08.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:26:08.602 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T13:26:08.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:08.701 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:08.485+0000 7f5319605200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:26:08.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:08.724 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:26:08.725 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T13:26:08.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:08.864 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:26:08.864 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T13:26:08.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T13:26:08.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:26:08.864 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T13:26:08.889 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:08.514+0000 7f874eb7d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:26:08.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:08.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:08.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:26:08.946 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:26:08.961 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:26:08.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:26:08.962 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:26:09.019 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T13:26:09.019 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T13:26:09.019 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/8a9db0d3-907a-4696-b98e-19a60db0c1d3 2024-04-25T13:26:09.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:09.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/8a9db0d3-907a-4696-b98e-19a60db0c1d3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:26:09.081 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:08.708+0000 7f5319605200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T13:26:09.134 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:08.802+0000 7f84c66fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:26:09.134 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:08.892+0000 7f84c66fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:26:09.158 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:26:09.158 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:26:09.159 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-25T13:26:09.269 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T13:26:09.269 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T13:26:09.269 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-25T13:26:09.270 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.44:3300,172.21.15.69:3300" 2024-04-25T13:26:09.270 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:3300,172.21.15.44:3300,172.21.15.69:3300:/volumes/qa/sv_1/8a9db0d3-907a-4696-b98e-19a60db0c1d3 2024-04-25T13:26:09.270 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=d766aeb0-0305-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T13:26:09.270 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T13:26:09.270 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-25T13:26:09.271 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T13:26:09.271 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T13:26:09.271 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T13:26:09.273 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T13:26:09.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:26:09.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:26:09.334 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:09.281+0000 7f5319605200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:26:09.339 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:26:09.385 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:09.134+0000 7f84c66fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:26:09.385 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:09.266+0000 7f84c66fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:26:09.385 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:09.359+0000 7f84c66fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:26:09.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.059+0000 7f874eb7d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:26:09.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.155+0000 7f874eb7d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:26:09.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:26:09.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:09.438 INFO:teuthology.orchestra.run.smithi037.stdout:client24887 2024-04-25T13:26:09.438 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:26:09.438 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:26:09.455 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:26:09.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:26:09.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:09.554 INFO:teuthology.orchestra.run.smithi037.stdout:client24887 2024-04-25T13:26:09.554 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T13:26:09.602 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:09.381+0000 7f5319605200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:26:09.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T13:26:09.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:09.612 INFO:teuthology.orchestra.run.smithi037.stdout:client24887 2024-04-25T13:26:09.613 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d766aeb0-0305-11ef-bc93-c7b262605968.client24887/status') 2024-04-25T13:26:09.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:26:09.672 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24887 (3)172.21.15.37:0/243061731 2024-04-25T13:26:09.672 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-25T13:26:09.672 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-25T13:26:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-25T13:26:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 2.5327e-05 s, 2.8 MB/s 2024-04-25T13:26:09.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:26:09.681 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T13:26:09.691 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T13:26:09.710 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:26:09.710 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:26:09.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T13:26:09.767 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.408+0000 7f874eb7d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:26:09.768 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.547+0000 7f874eb7d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:26:09.768 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.647+0000 7f874eb7d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:26:09.768 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:09.766+0000 7f874eb7d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:09.771 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:09.474+0000 7f84c66fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.816+0000 7f71ba643640 1 -- 172.21.15.37:0/1824384400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4100e60 msgr2=0x7f71b4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.816+0000 7f71ba643640 1 --2- 172.21.15.37:0/1824384400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4100e60 0x7f71b4101280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009900 tx=0x7f71a402a410 comp rx=0 tx=0).stop 2024-04-25T13:26:09.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 -- 172.21.15.37:0/1824384400 shutdown_connections 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 --2- 172.21.15.37:0/1824384400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71b4102a00 0x7f71b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 --2- 172.21.15.37:0/1824384400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b4102040 0x7f71b41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 --2- 172.21.15.37:0/1824384400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4100e60 0x7f71b4101280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 -- 172.21.15.37:0/1824384400 >> 172.21.15.37:0/1824384400 conn(0x7f71b40fc420 msgr2=0x7f71b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 -- 172.21.15.37:0/1824384400 shutdown_connections 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 -- 172.21.15.37:0/1824384400 wait complete. 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 Processor -- start 2024-04-25T13:26:09.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 -- start start 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.817+0000 7f71ba643640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71b4100e60 0x7f71b4198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 0x7f71b4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b4102a00 0x7f71b419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410c9f0 con 0x7f71b4102040 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410c870 con 0x7f71b4102a00 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f71b410cb70 con 0x7f71b4100e60 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 0x7f71b4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71b4100e60 0x7f71b4198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b8be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b4102a00 0x7f71b419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 0x7f71b4198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57986/0 (socket says 172.21.15.37:57986) 2024-04-25T13:26:09.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 -- 172.21.15.37:0/1662902535 learned_addr learned my addr 172.21.15.37:0/1662902535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71b4100e60 msgr2=0x7f71b4198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71b4100e60 0x7f71b4198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b4102a00 msgr2=0x7f71b419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71b4102a00 0x7f71b419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419fe60 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b37fe640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 0x7f71b4198780 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e9f0 tx=0x7f71a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a800cd80 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71a80108e0 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b41a0150 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8018980 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.818+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a06f0 con 0x7f71b4102040 2024-04-25T13:26:09.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.819+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b41a0950 con 0x7f71b4102040 2024-04-25T13:26:09.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.821+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f71a8010430 con 0x7f71b4102040 2024-04-25T13:26:09.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.822+0000 7f71b17fa640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:09.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.822+0000 7f71b3fff640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 msgr2=0x7f719c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:09.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.822+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8269+0+0 (secure 0 0 0) 0x7f71a80d07a0 con 0x7f71b4102040 2024-04-25T13:26:09.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.822+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:26:09.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:09.827+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71a809cab0 con 0x7f71b4102040 2024-04-25T13:26:09.907 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:09.650+0000 7f5319605200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:26:09.907 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:09.797+0000 7f5319605200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:26:09.907 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:09.906+0000 7f5319605200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:26:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.023+0000 7f71b3fff640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 msgr2=0x7f719c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:10.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.023+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:26:10.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.039+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f71b4069eb0 con 0x7f719c0a3c70 2024-04-25T13:26:10.135 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:09.771+0000 7f84c66fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:26:10.346 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:10.032+0000 7f5319605200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:26:10.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:10.078+0000 7f874eb7d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:26:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.424+0000 7f71b3fff640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 msgr2=0x7f719c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.424+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T13:26:10.601 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:10.346+0000 7f5319605200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:26:10.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:10.368+0000 7f84c66fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:26:10.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z[34406]: 2024-04-25T13:26:10.580+0000 7f84c66fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:26:10.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:10.643+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f71a8099120 con 0x7f71b4102040 2024-04-25T13:26:10.935 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:10.710+0000 7f874eb7d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:26:10.935 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: Standby manager daemon z restarted 2024-04-25T13:26:10.935 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: Standby manager daemon z started 2024-04-25T13:26:10.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:26:10.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:10.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:26:10.936 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: Standby manager daemon z restarted 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: Standby manager daemon z started 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:26:10.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: Standby manager daemon z restarted 2024-04-25T13:26:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: Standby manager daemon z started 2024-04-25T13:26:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T13:26:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T13:26:11.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:10 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.69:0/248885391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:11.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:11.225+0000 7f71b3fff640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 msgr2=0x7f719c0a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2819711067 2024-04-25T13:26:11.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:11.225+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T13:26:11.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:11.239+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f71a8098800 con 0x7f71b4102040 2024-04-25T13:26:11.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:11.239+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 msgr2=0x7f719c0a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T13:26:11.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:11.239+0000 7f71b17fa640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2819711067,v1:172.21.15.37:6801/2819711067] conn(0x7f719c0a3c70 0x7f719c0a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:11.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:10.993+0000 7f5319605200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:26:11.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.226+0000 7f5319605200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:26:11.389 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:26:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y[33986]: 2024-04-25T13:26:10.934+0000 7f874eb7d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f5278f63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f5278f63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f5278f63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f5278f63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f5278f63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f527b7a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f527b7a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f527b7a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f527b7a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:11.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:26:11.482+0000 7f527b7a8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:26:12.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: Standby manager daemon y restarted 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: Standby manager daemon y started 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: Active manager daemon x restarted 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: Activating manager daemon x 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: mgrmap e21: x(active, starting, since 0.0126218s), standbys: y, z 2024-04-25T13:26:12.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:26:12.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:26:12.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: Manager daemon x is now available 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:26:12.105 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:11 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:26:12.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:26:12.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: Standby manager daemon y restarted 2024-04-25T13:26:12.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: Standby manager daemon y started 2024-04-25T13:26:12.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:26:12.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:12.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: Active manager daemon x restarted 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: Activating manager daemon x 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: mgrmap e21: x(active, starting, since 0.0126218s), standbys: y, z 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:26:12.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:26:12.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:26:12.139 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: Manager daemon x is now available 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:26:12.140 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:11 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: Standby manager daemon y restarted 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: Standby manager daemon y started 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.? 172.21.15.44:0/274393984' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: Active manager daemon x restarted 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: Activating manager daemon x 2024-04-25T13:26:12.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: mgrmap e21: x(active, starting, since 0.0126218s), standbys: y, z 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T13:26:12.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T13:26:12.143 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T13:26:12.144 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: Manager daemon x is now available 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T13:26:12.145 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:11 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T13:26:12.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.244+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mon.0 v2:172.21.15.37:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f71a8082160 con 0x7f71b4102040 2024-04-25T13:26:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.245+0000 7f71b17fa640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f719c0353d0 0x7f719c0357b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.245+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f719c045420 con 0x7f719c0353d0 2024-04-25T13:26:12.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.248+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f719c0353d0 0x7f719c0357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.250+0000 7f71b3fff640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f719c0353d0 0x7f719c0357b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009230 tx=0x7f71a40314a0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.317+0000 7f71b17fa640 1 -- 172.21.15.37:0/1662902535 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f719c045420 con 0x7f719c0353d0 2024-04-25T13:26:12.318 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f719c0353d0 msgr2=0x7f719c0357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f719c0353d0 0x7f719c0357b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009230 tx=0x7f71a40314a0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 msgr2=0x7f71b4198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 --2- 172.21.15.37:0/1662902535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71b4102040 0x7f71b4198780 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71a800e9f0 tx=0x7f71a800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71b8be0640 1 -- 172.21.15.37:0/1662902535 reap_dead start 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 shutdown_connections 2024-04-25T13:26:12.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 >> 172.21.15.37:0/1662902535 conn(0x7f71b40fc420 msgr2=0x7f71b40697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:12.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 shutdown_connections 2024-04-25T13:26:12.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.320+0000 7f71ba643640 1 -- 172.21.15.37:0/1662902535 wait complete. 2024-04-25T13:26:12.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T13:26:12.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.438+0000 7f8924520640 1 -- 172.21.15.37:0/1306424964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c103b70 msgr2=0x7f891c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.438+0000 7f8924520640 1 --2- 172.21.15.37:0/1306424964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c103b70 0x7f891c10a400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8914009ca0 tx=0x7f891402a8c0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- 172.21.15.37:0/1306424964 shutdown_connections 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- 172.21.15.37:0/1306424964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c103b70 0x7f891c10a400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- 172.21.15.37:0/1306424964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c1031b0 0x7f891c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- 172.21.15.37:0/1306424964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c101fc0 0x7f891c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- 172.21.15.37:0/1306424964 >> 172.21.15.37:0/1306424964 conn(0x7f891c0fd530 msgr2=0x7f891c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- 172.21.15.37:0/1306424964 shutdown_connections 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- 172.21.15.37:0/1306424964 wait complete. 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 Processor -- start 2024-04-25T13:26:12.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- start start 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c101fc0 0x7f891c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 0x7f891c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f891c10db00 con 0x7f891c1031b0 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f891c10d980 con 0x7f891c103b70 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8924520640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f891c10dc80 con 0x7f891c101fc0 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.439+0000 7f8921abb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f89222bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c101fc0 0x7f891c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58084/0 (socket says 172.21.15.37:58084) 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 -- 172.21.15.37:0/624491387 learned_addr learned my addr 172.21.15.37:0/624491387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:12.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8922abd640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 0x7f891c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 -- 172.21.15.37:0/624491387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c101fc0 msgr2=0x7f891c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c101fc0 0x7f891c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 -- 172.21.15.37:0/624491387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 msgr2=0x7f891c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 0x7f891c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 -- 172.21.15.37:0/624491387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c19fdd0 con 0x7f891c1031b0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8922abd640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 0x7f891c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8921abb640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f890c00e9f0 tx=0x7f890c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c00cd80 con 0x7f891c1031b0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890c0054d0 con 0x7f891c1031b0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c1a00c0 con 0x7f891c1031b0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c010690 con 0x7f891c1031b0 2024-04-25T13:26:12.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.440+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c1a0610 con 0x7f891c1031b0 2024-04-25T13:26:12.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.441+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c109360 con 0x7f891c1031b0 2024-04-25T13:26:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.442+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f890c004e90 con 0x7f891c1031b0 2024-04-25T13:26:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.443+0000 7f89137fe640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 0x7f89080a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.443+0000 7f89222bc640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 0x7f89080a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.443+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8527+0+0 (secure 0 0 0) 0x7f890c0d0560 con 0x7f891c1031b0 2024-04-25T13:26:12.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.443+0000 7f89222bc640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 0x7f89080a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f891c103010 tx=0x7f891800b390 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:12.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.447+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f890c09d110 con 0x7f891c1031b0 2024-04-25T13:26:12.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.663+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f891c062430 con 0x7f89080a3a10 2024-04-25T13:26:12.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.724+0000 7f89137fe640 1 -- 172.21.15.37:0/624491387 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f891c062430 con 0x7f89080a3a10 2024-04-25T13:26:12.726 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-04-25T13:26:12.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 msgr2=0x7f89080a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 0x7f89080a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f891c103010 tx=0x7f891800b390 comp rx=0 tx=0).stop 2024-04-25T13:26:12.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 msgr2=0x7f891c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f890c00e9f0 tx=0x7f890c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 shutdown_connections 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f89080a3a10 0x7f89080a5ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f891c103b70 0x7f891c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f891c1031b0 0x7f891c1986f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 --2- 172.21.15.37:0/624491387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f891c101fc0 0x7f891c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 >> 172.21.15.37:0/624491387 conn(0x7f891c0fd530 msgr2=0x7f891c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 shutdown_connections 2024-04-25T13:26:12.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.726+0000 7f8924520640 1 -- 172.21.15.37:0/624491387 wait complete. 2024-04-25T13:26:12.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 -- 172.21.15.37:0/2771537872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17401031b0 msgr2=0x7f1740103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 --2- 172.21.15.37:0/2771537872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17401031b0 0x7f1740103630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1734009900 tx=0x7f173402a410 comp rx=0 tx=0).stop 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 -- 172.21.15.37:0/2771537872 shutdown_connections 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 --2- 172.21.15.37:0/2771537872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 0x7f174010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 --2- 172.21.15.37:0/2771537872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17401031b0 0x7f1740103630 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 --2- 172.21.15.37:0/2771537872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1740101fc0 0x7f17401023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 -- 172.21.15.37:0/2771537872 >> 172.21.15.37:0/2771537872 conn(0x7f17400fd530 msgr2=0x7f17400ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 -- 172.21.15.37:0/2771537872 shutdown_connections 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.841+0000 7f1745568640 1 -- 172.21.15.37:0/2771537872 wait complete. 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 Processor -- start 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 -- start start 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 0x7f174019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 0x7f17401a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f174010db00 con 0x7f1740101fc0 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f174010d980 con 0x7f1740103b70 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f1745568640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f174010dc80 con 0x7f17401031b0 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.842+0000 7f173effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58088/0 (socket says 172.21.15.37:58088) 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 -- 172.21.15.37:0/3181455778 learned_addr learned my addr 172.21.15.37:0/3181455778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173e7fc640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 0x7f174019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173f7fe640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 0x7f17401a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 -- 172.21.15.37:0/3181455778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 msgr2=0x7f174019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 0x7f174019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 -- 172.21.15.37:0/3181455778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 msgr2=0x7f17401a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 0x7f17401a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 -- 172.21.15.37:0/3181455778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401a2020 con 0x7f1740101fc0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173e7fc640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 0x7f174019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f173effd640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9c0 tx=0x7f173000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173000cd80 con 0x7f1740101fc0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401a2310 con 0x7f1740101fc0 2024-04-25T13:26:12.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17300035f0 con 0x7f1740101fc0 2024-04-25T13:26:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730010690 con 0x7f1740101fc0 2024-04-25T13:26:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.843+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401a27e0 con 0x7f1740101fc0 2024-04-25T13:26:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.844+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17401a2aa0 con 0x7f1740101fc0 2024-04-25T13:26:12.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.845+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1730004000 con 0x7f1740101fc0 2024-04-25T13:26:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.846+0000 7f1727fff640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 0x7f17280a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.846+0000 7f173e7fc640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 0x7f17280a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.846+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8527+0+0 (secure 0 0 0) 0x7f17300d0650 con 0x7f1740101fc0 2024-04-25T13:26:12.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.846+0000 7f173e7fc640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 0x7f17280a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f17340314a0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:12.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:12.852+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f173009d1c0 con 0x7f1740101fc0 2024-04-25T13:26:12.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:12 smithi069 ceph-mon[32534]: [25/Apr/2024:13:26:11] ENGINE Bus STARTING 2024-04-25T13:26:12.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:12 smithi069 ceph-mon[32534]: [25/Apr/2024:13:26:11] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T13:26:12.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:12 smithi069 ceph-mon[32534]: mgrmap e22: x(active, since 1.02147s), standbys: y, z 2024-04-25T13:26:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:12 smithi044 ceph-mon[32590]: [25/Apr/2024:13:26:11] ENGINE Bus STARTING 2024-04-25T13:26:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:12 smithi044 ceph-mon[32590]: [25/Apr/2024:13:26:11] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T13:26:12.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:12 smithi044 ceph-mon[32590]: mgrmap e22: x(active, since 1.02147s), standbys: y, z 2024-04-25T13:26:13.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.063+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f17400758a0 con 0x7f17280a3990 2024-04-25T13:26:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.070+0000 7f1727fff640 1 -- 172.21.15.37:0/3181455778 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f17400758a0 con 0x7f17280a3990 2024-04-25T13:26:13.072 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:26:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 msgr2=0x7f17280a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 0x7f17280a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1734000c00 tx=0x7f17340314a0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 msgr2=0x7f174019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f173000e9c0 tx=0x7f173000ee80 comp rx=0 tx=0).stop 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 shutdown_connections 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f17280a3990 0x7f17280a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1740103b70 0x7f17401a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17401031b0 0x7f174019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 --2- 172.21.15.37:0/3181455778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1740101fc0 0x7f174019a400 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.073+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 >> 172.21.15.37:0/3181455778 conn(0x7f17400fd530 msgr2=0x7f1740074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:13.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.074+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 shutdown_connections 2024-04-25T13:26:13.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.074+0000 7f1745568640 1 -- 172.21.15.37:0/3181455778 wait complete. 2024-04-25T13:26:13.085 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:12 smithi037 ceph-mon[29204]: [25/Apr/2024:13:26:11] ENGINE Bus STARTING 2024-04-25T13:26:13.085 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:12 smithi037 ceph-mon[29204]: [25/Apr/2024:13:26:11] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T13:26:13.085 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:12 smithi037 ceph-mon[29204]: mgrmap e22: x(active, since 1.02147s), standbys: y, z 2024-04-25T13:26:13.086 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/2481195493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 msgr2=0x7f5a98074db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/2481195493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a98074db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009900 tx=0x7f5a8002a480 comp rx=0 tx=0).stop 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/2481195493 shutdown_connections 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/2481195493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98109e60 0x7f5a9810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/2481195493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a98074db0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/2481195493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98075f30 0x7f5a98074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/2481195493 >> 172.21.15.37:0/2481195493 conn(0x7f5a980fd540 msgr2=0x7f5a980ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/2481195493 shutdown_connections 2024-04-25T13:26:13.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.204+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/2481195493 wait complete. 2024-04-25T13:26:13.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 Processor -- start 2024-04-25T13:26:13.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 -- start start 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 0x7f5a9819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 0x7f5a981a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9810fd60 con 0x7f5a98074950 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9810fbe0 con 0x7f5a98109e60 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.205+0000 7f5a9e9ed640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9810fee0 con 0x7f5a98075f30 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a977fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 0x7f5a9819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a9cf8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 0x7f5a981a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58116/0 (socket says 172.21.15.37:58116) 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 -- 172.21.15.37:0/190120669 learned_addr learned my addr 172.21.15.37:0/190120669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 -- 172.21.15.37:0/190120669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 msgr2=0x7f5a9819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 0x7f5a9819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 -- 172.21.15.37:0/190120669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 msgr2=0x7f5a981a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 0x7f5a981a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 -- 172.21.15.37:0/190120669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a981a2040 con 0x7f5a98074950 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a9cf8a640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 0x7f5a981a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a977fe640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 0x7f5a9819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a97fff640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800cd80 con 0x7f5a98074950 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a880108e0 con 0x7f5a98074950 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a981a2330 con 0x7f5a98074950 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.206+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88018980 con 0x7f5a98074950 2024-04-25T13:26:13.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.207+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a981a2850 con 0x7f5a98074950 2024-04-25T13:26:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.208+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a98108c20 con 0x7f5a98074950 2024-04-25T13:26:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.209+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5a88010430 con 0x7f5a98074950 2024-04-25T13:26:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.209+0000 7f5a957fa640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 0x7f5a840a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:26:13.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.210+0000 7f5a977fe640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 0x7f5a840a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:26:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.210+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8527+0+0 (secure 0 0 0) 0x7f5a880d03f0 con 0x7f5a98074950 2024-04-25T13:26:13.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.210+0000 7f5a977fe640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 0x7f5a840a5e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a80031500 tx=0x7f5a80005a20 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:26:13.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.214+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a8809c950 con 0x7f5a98074950 2024-04-25T13:26:13.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.470+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5a98062390 con 0x7f5a840a3990 2024-04-25T13:26:13.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.476+0000 7f5a957fa640 1 -- 172.21.15.37:0/190120669 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5a98062390 con 0x7f5a840a3990 2024-04-25T13:26:13.478 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-25T13:26:13.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 msgr2=0x7f5a840a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 0x7f5a840a5e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a80031500 tx=0x7f5a80005a20 comp rx=0 tx=0).stop 2024-04-25T13:26:13.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 msgr2=0x7f5a9819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:26:13.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800eeb0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 shutdown_connections 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a840a3990 0x7f5a840a5e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a98109e60 0x7f5a981a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a98075f30 0x7f5a9819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 --2- 172.21.15.37:0/190120669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98074950 0x7f5a9819a470 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 >> 172.21.15.37:0/190120669 conn(0x7f5a980fd540 msgr2=0x7f5a98105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 shutdown_connections 2024-04-25T13:26:13.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:26:13.479+0000 7f5a9e9ed640 1 -- 172.21.15.37:0/190120669 wait complete. 2024-04-25T13:26:13.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T13:26:13.520 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T13:26:13.531 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T13:26:13.589 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:26:13.589 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T13:26:13.591 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T13:26:13.604 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:26:13.851 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=12044fe366324769b217fec27d767ecc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_4d7908725372ec8e17504b848bfa122dd1d43386/qa/tasks/quiescer.py", line 387, in task cluster_name = config.get('cluster', 'ceph') AttributeError: 'NoneType' object has no attribute 'get' 2024-04-25T13:26:13.853 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T13:26:13.864 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T13:26:13.873 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:26:13.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: [25/Apr/2024:13:26:11] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T13:26:13.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: [25/Apr/2024:13:26:11] ENGINE Bus STARTED 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:13 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:13.886 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T13:26:13.897 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T13:26:13.897 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T13:26:13.897 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:26:13.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T13:26:13.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T13:26:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: [25/Apr/2024:13:26:11] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T13:26:14.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: [25/Apr/2024:13:26:11] ENGINE Bus STARTED 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:13 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: [25/Apr/2024:13:26:11] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: [25/Apr/2024:13:26:11] ENGINE Bus STARTED 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:14.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:13 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:14 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:14 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:15.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:14 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:26:16.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:26:16.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:26:16.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:15 smithi037 ceph-mon[29204]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:26:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:26:16.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:26:16.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:15 smithi069 ceph-mon[32534]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:26:16.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Updating smithi044:/etc/ceph/ceph.conf 2024-04-25T13:26:16.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:15 smithi044 ceph-mon[32590]: Updating smithi069:/etc/ceph/ceph.conf 2024-04-25T13:26:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:17.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:17.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:17.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:17.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:17.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:17.138 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-04-25T13:26:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: Updating smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:17.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:26:17.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:26:17.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:26:17.142 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:26:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:18 smithi037 ceph-mon[29204]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-25T13:26:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:18 smithi069 ceph-mon[32534]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-25T13:26:19.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:18 smithi044 ceph-mon[32590]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 65 op/s 2024-04-25T13:26:21.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:20 smithi037 ceph-mon[29204]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:26:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:20 smithi069 ceph-mon[32534]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:26:21.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:20 smithi044 ceph-mon[32590]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-04-25T13:26:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:22 smithi037 ceph-mon[29204]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:26:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:22 smithi069 ceph-mon[32534]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:26:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:22 smithi044 ceph-mon[32590]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-04-25T13:26:25.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:24 smithi037 ceph-mon[29204]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:26:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:24 smithi069 ceph-mon[32534]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:26:25.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:24 smithi044 ceph-mon[32590]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T13:26:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:26 smithi037 ceph-mon[29204]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:26 smithi069 ceph-mon[32534]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:26 smithi044 ceph-mon[32590]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:28 smithi037 ceph-mon[29204]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:28 smithi069 ceph-mon[32534]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:28 smithi044 ceph-mon[32590]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T13:26:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:30 smithi037 ceph-mon[29204]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T13:26:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:30 smithi069 ceph-mon[32534]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T13:26:31.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:30 smithi044 ceph-mon[32590]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T13:26:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:32 smithi037 ceph-mon[29204]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:32 smithi069 ceph-mon[32534]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:33.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:32 smithi044 ceph-mon[32590]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:34 smithi037 ceph-mon[29204]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:34 smithi069 ceph-mon[32534]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:34 smithi044 ceph-mon[32590]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:36 smithi037 ceph-mon[29204]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:36 smithi069 ceph-mon[32534]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:36 smithi044 ceph-mon[32590]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:38 smithi037 ceph-mon[29204]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:38 smithi069 ceph-mon[32534]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:38 smithi044 ceph-mon[32590]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:40 smithi037 ceph-mon[29204]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:40 smithi069 ceph-mon[32534]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:40 smithi044 ceph-mon[32590]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:42 smithi037 ceph-mon[29204]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:42 smithi069 ceph-mon[32534]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:42 smithi044 ceph-mon[32590]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:44 smithi037 ceph-mon[29204]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:44 smithi069 ceph-mon[32534]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:45.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:44 smithi044 ceph-mon[32590]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:46 smithi037 ceph-mon[29204]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:46 smithi069 ceph-mon[32534]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:46 smithi044 ceph-mon[32590]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:48 smithi037 ceph-mon[29204]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:48 smithi069 ceph-mon[32534]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:48 smithi044 ceph-mon[32590]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:50 smithi037 ceph-mon[29204]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:50 smithi069 ceph-mon[32534]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:50 smithi044 ceph-mon[32590]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:52 smithi037 ceph-mon[29204]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:52 smithi069 ceph-mon[32534]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:52 smithi044 ceph-mon[32590]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:54 smithi037 ceph-mon[29204]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:55.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:54 smithi069 ceph-mon[32534]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:54 smithi044 ceph-mon[32590]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:26:57.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:56 smithi037 ceph-mon[29204]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:56 smithi069 ceph-mon[32534]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:56 smithi044 ceph-mon[32590]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:26:58 smithi037 ceph-mon[29204]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:26:58 smithi069 ceph-mon[32534]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:26:59.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:26:58 smithi044 ceph-mon[32590]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:01.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-mon[29204]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:01.102 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:27:00.736+0000 7f5259ae5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:27:01.103 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:27:00.736+0000 7f5259ae5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:27:01.103 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:27:00.736+0000 7f5259ae5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:27:01.103 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:27:00.736+0000 7f5259ae5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:27:01.103 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:27:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:27:00.736+0000 7f5259ae5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T13:27:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:00 smithi069 ceph-mon[32534]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:01.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:00 smithi044 ceph-mon[32590]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:02 smithi037 ceph-mon[29204]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:02 smithi037 ceph-mon[29204]: mgrmap e25: x(active, since 50s), standbys: y, z 2024-04-25T13:27:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:02 smithi069 ceph-mon[32534]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:02 smithi069 ceph-mon[32534]: mgrmap e25: x(active, since 50s), standbys: y, z 2024-04-25T13:27:03.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:02 smithi044 ceph-mon[32590]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:03.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:02 smithi044 ceph-mon[32590]: mgrmap e25: x(active, since 50s), standbys: y, z 2024-04-25T13:27:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:04 smithi037 ceph-mon[29204]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:04 smithi069 ceph-mon[32534]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:04 smithi044 ceph-mon[32590]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:06 smithi037 ceph-mon[29204]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:06 smithi069 ceph-mon[32534]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:06 smithi044 ceph-mon[32590]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:08 smithi037 ceph-mon[29204]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:08 smithi069 ceph-mon[32534]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:08 smithi044 ceph-mon[32590]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:10 smithi037 ceph-mon[29204]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:10 smithi069 ceph-mon[32534]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:10 smithi044 ceph-mon[32590]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:12 smithi037 ceph-mon[29204]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:12 smithi069 ceph-mon[32534]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:12 smithi044 ceph-mon[32590]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:27:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:14 smithi037 ceph-mon[29204]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:14 smithi069 ceph-mon[32534]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:14 smithi044 ceph-mon[32590]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:27:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:16 smithi037 ceph-mon[29204]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:27:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:16 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:16 smithi069 ceph-mon[32534]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:27:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:16 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:16 smithi044 ceph-mon[32590]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:27:17.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:16 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:27:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:18.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:18.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:18.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:18.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:18.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:27:18.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:27:18.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:27:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:18 smithi037 ceph-mon[29204]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:18 smithi069 ceph-mon[32534]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:19.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:18 smithi044 ceph-mon[32590]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:27:21.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:20 smithi037 ceph-mon[29204]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:20 smithi069 ceph-mon[32534]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:21.141 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:20 smithi044 ceph-mon[32590]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:22 smithi037 ceph-mon[29204]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:22 smithi069 ceph-mon[32534]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:23.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:22 smithi044 ceph-mon[32590]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:25.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:24 smithi037 ceph-mon[29204]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:24 smithi069 ceph-mon[32534]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:25.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:24 smithi044 ceph-mon[32590]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:26 smithi037 ceph-mon[29204]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:26 smithi069 ceph-mon[32534]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:27.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:26 smithi044 ceph-mon[32590]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:28 smithi037 ceph-mon[29204]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:28 smithi069 ceph-mon[32534]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:28 smithi044 ceph-mon[32590]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:30 smithi037 ceph-mon[29204]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:30 smithi069 ceph-mon[32534]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:31.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:30 smithi044 ceph-mon[32590]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:32 smithi037 ceph-mon[29204]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:32 smithi069 ceph-mon[32534]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:33.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:32 smithi044 ceph-mon[32590]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:34 smithi037 ceph-mon[29204]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:34 smithi069 ceph-mon[32534]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:34 smithi044 ceph-mon[32590]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:36 smithi037 ceph-mon[29204]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:36 smithi069 ceph-mon[32534]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:36 smithi044 ceph-mon[32590]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:38 smithi069 ceph-mon[32534]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:38 smithi044 ceph-mon[32590]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:38 smithi037 ceph-mon[29204]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:40 smithi069 ceph-mon[32534]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:40 smithi044 ceph-mon[32590]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:41.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:40 smithi037 ceph-mon[29204]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:42 smithi069 ceph-mon[32534]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:43.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:42 smithi044 ceph-mon[32590]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:43.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:42 smithi037 ceph-mon[29204]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:44 smithi069 ceph-mon[32534]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:44 smithi069 ceph-mon[32534]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:27:45.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:44 smithi044 ceph-mon[32590]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:45.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:44 smithi044 ceph-mon[32590]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:27:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:44 smithi037 ceph-mon[29204]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:44 smithi037 ceph-mon[29204]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T13:27:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:46 smithi069 ceph-mon[32534]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:47.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:46 smithi044 ceph-mon[32590]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:46 smithi037 ceph-mon[29204]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:48 smithi069 ceph-mon[32534]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:48 smithi044 ceph-mon[32590]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:49.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:48 smithi037 ceph-mon[29204]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:51.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:50 smithi069 ceph-mon[32534]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:51.156 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:50 smithi044 ceph-mon[32590]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:50 smithi037 ceph-mon[29204]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:52 smithi069 ceph-mon[32534]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:52 smithi044 ceph-mon[32590]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:53.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:52 smithi037 ceph-mon[29204]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:54 smithi069 ceph-mon[32534]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:54 smithi044 ceph-mon[32590]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:54 smithi037 ceph-mon[29204]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T13:27:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:56 smithi069 ceph-mon[32534]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:56 smithi044 ceph-mon[32590]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:57.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:56 smithi037 ceph-mon[29204]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:27:59.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:27:58 smithi044 ceph-mon[32590]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:27:58 smithi037 ceph-mon[29204]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:27:59.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:27:58 smithi069 ceph-mon[32534]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:01.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:00 smithi037 ceph-mon[29204]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:01.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:00 smithi069 ceph-mon[32534]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:01.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:00 smithi044 ceph-mon[32590]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:02 smithi037 ceph-mon[29204]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:03.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:02 smithi069 ceph-mon[32534]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:03.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:02 smithi044 ceph-mon[32590]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:05.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:04 smithi037 ceph-mon[29204]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:05.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:04 smithi069 ceph-mon[32534]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:05.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:04 smithi044 ceph-mon[32590]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:06 smithi037 ceph-mon[29204]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:07.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:06 smithi069 ceph-mon[32534]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:07.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:06 smithi044 ceph-mon[32590]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:08 smithi037 ceph-mon[29204]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:09.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:08 smithi069 ceph-mon[32534]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:09.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:08 smithi044 ceph-mon[32590]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:10 smithi037 ceph-mon[29204]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:10 smithi069 ceph-mon[32534]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:11.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:10 smithi044 ceph-mon[32590]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:12 smithi037 ceph-mon[29204]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:12 smithi069 ceph-mon[32534]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:13.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:12 smithi044 ceph-mon[32590]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:28:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:14 smithi037 ceph-mon[29204]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:14 smithi069 ceph-mon[32534]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:14 smithi044 ceph-mon[32590]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:28:17.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:16 smithi037 ceph-mon[29204]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:28:17.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:16 smithi069 ceph-mon[32534]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:28:17.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:16 smithi044 ceph-mon[32590]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:28:18.285 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:17 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:18.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:17 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:18.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:17 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:28:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:19.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:19.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:18 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:19.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:19.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:18 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:28:19.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:18 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:28:21.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:20 smithi037 ceph-mon[29204]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:21.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:20 smithi069 ceph-mon[32534]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:21.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:20 smithi044 ceph-mon[32590]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:23.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:22 smithi037 ceph-mon[29204]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:23.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:22 smithi069 ceph-mon[32534]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:23.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:22 smithi044 ceph-mon[32590]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:24.291 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:23 smithi044 ceph-mon[32590]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:24.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:23 smithi037 ceph-mon[29204]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:24.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:23 smithi069 ceph-mon[32534]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:26.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:26 smithi037 ceph-mon[29204]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:26.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:26 smithi069 ceph-mon[32534]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:26.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:26 smithi044 ceph-mon[32590]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:28.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:28 smithi037 ceph-mon[29204]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:28.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:28 smithi069 ceph-mon[32534]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:28.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:28 smithi044 ceph-mon[32590]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:30.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:30 smithi037 ceph-mon[29204]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:30.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:30 smithi069 ceph-mon[32534]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:30.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:30 smithi044 ceph-mon[32590]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:32.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:32 smithi037 ceph-mon[29204]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:32.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:32 smithi069 ceph-mon[32534]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:32.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:32 smithi044 ceph-mon[32590]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:34.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:34 smithi037 ceph-mon[29204]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:34.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:34 smithi069 ceph-mon[32534]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:34.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:34 smithi044 ceph-mon[32590]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:36.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:36 smithi037 ceph-mon[29204]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:36.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:36 smithi069 ceph-mon[32534]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:36.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:36 smithi044 ceph-mon[32590]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:38.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:38 smithi069 ceph-mon[32534]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:38.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:38 smithi044 ceph-mon[32590]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:38 smithi037 ceph-mon[29204]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:40.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:40 smithi069 ceph-mon[32534]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:40.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:40 smithi044 ceph-mon[32590]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:40 smithi037 ceph-mon[29204]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:42.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:42 smithi069 ceph-mon[32534]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:42.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:42 smithi044 ceph-mon[32590]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:42.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:42 smithi037 ceph-mon[29204]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:44.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:44 smithi069 ceph-mon[32534]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:44.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:44 smithi044 ceph-mon[32590]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:44 smithi037 ceph-mon[29204]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:46.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:46 smithi069 ceph-mon[32534]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:46.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:46 smithi044 ceph-mon[32590]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:46.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:46 smithi037 ceph-mon[29204]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:48.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:48 smithi069 ceph-mon[32534]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:48.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:48 smithi044 ceph-mon[32590]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:48.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:48 smithi037 ceph-mon[29204]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:50.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:50 smithi069 ceph-mon[32534]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:50.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:50 smithi044 ceph-mon[32590]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:50 smithi037 ceph-mon[29204]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:52.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:52 smithi069 ceph-mon[32534]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:52.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:52 smithi044 ceph-mon[32590]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:52.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:52 smithi037 ceph-mon[29204]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:54.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:54 smithi069 ceph-mon[32534]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:54.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:54 smithi044 ceph-mon[32590]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:54.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:54 smithi037 ceph-mon[29204]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:56.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:56 smithi069 ceph-mon[32534]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:56.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:56 smithi044 ceph-mon[32590]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:56 smithi037 ceph-mon[29204]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:28:58.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:28:58 smithi069 ceph-mon[32534]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:58.639 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:28:58 smithi044 ceph-mon[32590]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:28:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:28:58 smithi037 ceph-mon[29204]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:00.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:00 smithi044 ceph-mon[32590]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:00 smithi037 ceph-mon[29204]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:00 smithi069 ceph-mon[32534]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:02 smithi037 ceph-mon[29204]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:02 smithi069 ceph-mon[32534]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:02 smithi044 ceph-mon[32590]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:04.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:04 smithi037 ceph-mon[29204]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:04 smithi069 ceph-mon[32534]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:04.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:04 smithi044 ceph-mon[32590]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:06 smithi037 ceph-mon[29204]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:06 smithi069 ceph-mon[32534]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:06.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:06 smithi044 ceph-mon[32590]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:08.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:08 smithi037 ceph-mon[29204]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:08 smithi069 ceph-mon[32534]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:08 smithi044 ceph-mon[32590]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:10 smithi037 ceph-mon[29204]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:10.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:10 smithi069 ceph-mon[32534]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:10 smithi044 ceph-mon[32590]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:12 smithi037 ceph-mon[29204]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:12 smithi069 ceph-mon[32534]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:12 smithi044 ceph-mon[32590]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:29:14.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:14 smithi037 ceph-mon[29204]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:14 smithi069 ceph-mon[32534]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:14.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:14 smithi044 ceph-mon[32590]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:29:16.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:16 smithi037 ceph-mon[29204]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:29:16.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:16 smithi069 ceph-mon[32534]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:29:16.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:16 smithi044 ceph-mon[32590]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:29:18.687 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:18 smithi069 ceph-mon[32534]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:18 smithi037 ceph-mon[29204]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:18 smithi044 ceph-mon[32590]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:29:19.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:19 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:19.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:19 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:19.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:19 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:29:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:20 smithi037 ceph-mon[29204]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:20 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:20.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:20 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:20.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:20 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:29:20.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:20 smithi069 ceph-mon[32534]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:20.887 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:20 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:20.887 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:20 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:20.887 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:20 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:29:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:20 smithi044 ceph-mon[32590]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:20 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:29:20.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:20 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:29:20.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:20 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:29:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:22 smithi037 ceph-mon[29204]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:22 smithi069 ceph-mon[32534]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:22 smithi044 ceph-mon[32590]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:24.692 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:24 smithi044 ceph-mon[32590]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:24 smithi037 ceph-mon[29204]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:24 smithi069 ceph-mon[32534]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:26 smithi037 ceph-mon[29204]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:26 smithi069 ceph-mon[32534]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:26.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:26 smithi044 ceph-mon[32590]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:28 smithi037 ceph-mon[29204]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:28 smithi069 ceph-mon[32534]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:28.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:28 smithi044 ceph-mon[32590]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:30.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:30 smithi037 ceph-mon[29204]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:30.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:30 smithi069 ceph-mon[32534]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:30.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:30 smithi044 ceph-mon[32590]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:32.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:32 smithi037 ceph-mon[29204]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:32.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:32 smithi069 ceph-mon[32534]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:32.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:32 smithi044 ceph-mon[32590]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:34.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:34 smithi037 ceph-mon[29204]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:34.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:34 smithi069 ceph-mon[32534]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:34.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:34 smithi044 ceph-mon[32590]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:36 smithi037 ceph-mon[29204]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:36.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:36 smithi069 ceph-mon[32534]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:36.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:36 smithi044 ceph-mon[32590]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:38 smithi037 ceph-mon[29204]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:38.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:38 smithi069 ceph-mon[32534]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:38.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:38 smithi044 ceph-mon[32590]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:40 smithi037 ceph-mon[29204]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:40.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:40 smithi069 ceph-mon[32534]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:40.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:40 smithi044 ceph-mon[32590]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:42 smithi037 ceph-mon[29204]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:42 smithi069 ceph-mon[32534]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:42.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:42 smithi044 ceph-mon[32590]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:44 smithi037 ceph-mon[29204]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:44 smithi069 ceph-mon[32534]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:44.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:44 smithi044 ceph-mon[32590]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:46.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:46 smithi037 ceph-mon[29204]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:46.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:46 smithi069 ceph-mon[32534]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:46.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:46 smithi044 ceph-mon[32590]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:48.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:48 smithi037 ceph-mon[29204]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:48.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:48 smithi069 ceph-mon[32534]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:48.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:48 smithi044 ceph-mon[32590]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:50 smithi037 ceph-mon[29204]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:50 smithi069 ceph-mon[32534]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:50.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:50 smithi044 ceph-mon[32590]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:52.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:52 smithi037 ceph-mon[29204]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:52.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:52 smithi069 ceph-mon[32534]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:52.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:52 smithi044 ceph-mon[32590]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:54.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:54 smithi037 ceph-mon[29204]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:54.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:54 smithi069 ceph-mon[32534]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:54.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:54 smithi044 ceph-mon[32590]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:56 smithi037 ceph-mon[29204]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:56 smithi069 ceph-mon[32534]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:56.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:56 smithi044 ceph-mon[32590]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:29:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:29:58 smithi037 ceph-mon[29204]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:29:58 smithi069 ceph-mon[32534]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:29:58.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:29:58 smithi044 ceph-mon[32590]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:00 smithi037 ceph-mon[29204]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:00 smithi037 ceph-mon[29204]: overall HEALTH_OK 2024-04-25T13:30:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:00 smithi069 ceph-mon[32534]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:00 smithi069 ceph-mon[32534]: overall HEALTH_OK 2024-04-25T13:30:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:00 smithi044 ceph-mon[32590]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:00.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:00 smithi044 ceph-mon[32590]: overall HEALTH_OK 2024-04-25T13:30:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:02 smithi037 ceph-mon[29204]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:02 smithi069 ceph-mon[32534]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:02.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:02 smithi044 ceph-mon[32590]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:04.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:04 smithi037 ceph-mon[29204]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:04 smithi069 ceph-mon[32534]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:04.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:04 smithi044 ceph-mon[32590]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:06 smithi037 ceph-mon[29204]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:06 smithi069 ceph-mon[32534]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:06.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:06 smithi044 ceph-mon[32590]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:08 smithi037 ceph-mon[29204]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:08 smithi069 ceph-mon[32534]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:08 smithi044 ceph-mon[32590]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:10 smithi037 ceph-mon[29204]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:10.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:10 smithi069 ceph-mon[32534]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:10 smithi044 ceph-mon[32590]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:12 smithi037 ceph-mon[29204]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:12 smithi069 ceph-mon[32534]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:12.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:12 smithi044 ceph-mon[32590]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:30:14.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:14 smithi037 ceph-mon[29204]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:14 smithi069 ceph-mon[32534]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:14.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:14 smithi044 ceph-mon[32590]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:30:16.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:16 smithi037 ceph-mon[29204]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:30:16.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:16 smithi069 ceph-mon[32534]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:30:16.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:16 smithi044 ceph-mon[32590]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:30:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:18 smithi037 ceph-mon[29204]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:30:18.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:18 smithi069 ceph-mon[32534]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:30:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:18 smithi044 ceph-mon[32590]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:30:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:20 smithi037 ceph-mon[29204]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:20 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:30:20.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:20 smithi069 ceph-mon[32534]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:20.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:20 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:30:20.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:20 smithi044 ceph-mon[32590]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:20.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:20 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:30:21.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:30:21.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:30:21.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:30:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:30:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:30:21.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:30:21.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:30:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:30:21.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:30:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:22 smithi069 ceph-mon[32534]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:22 smithi044 ceph-mon[32590]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:22 smithi037 ceph-mon[29204]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:24 smithi069 ceph-mon[32534]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:24.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:24 smithi044 ceph-mon[32590]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:24.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:24 smithi037 ceph-mon[29204]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:26 smithi069 ceph-mon[32534]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:26.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:26 smithi044 ceph-mon[32590]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:26 smithi037 ceph-mon[29204]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:28 smithi069 ceph-mon[32534]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:28 smithi044 ceph-mon[32590]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:29.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:28 smithi037 ceph-mon[29204]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:30.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:30 smithi069 ceph-mon[32534]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:30.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:30 smithi044 ceph-mon[32590]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:30 smithi037 ceph-mon[29204]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:32.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:32 smithi069 ceph-mon[32534]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:32.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:32 smithi044 ceph-mon[32590]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:32 smithi037 ceph-mon[29204]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:34.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:34 smithi069 ceph-mon[32534]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:34.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:34 smithi044 ceph-mon[32590]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:34 smithi037 ceph-mon[29204]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:36.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:36 smithi069 ceph-mon[32534]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:36.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:36 smithi044 ceph-mon[32590]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:36 smithi037 ceph-mon[29204]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:38.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:38 smithi044 ceph-mon[32590]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:38 smithi037 ceph-mon[29204]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:38 smithi069 ceph-mon[32534]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:40 smithi044 ceph-mon[32590]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:40 smithi037 ceph-mon[29204]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:40 smithi069 ceph-mon[32534]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:42 smithi037 ceph-mon[29204]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:42 smithi069 ceph-mon[32534]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:43.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:42 smithi044 ceph-mon[32590]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:44 smithi037 ceph-mon[29204]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:44 smithi069 ceph-mon[32534]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:45.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:44 smithi044 ceph-mon[32590]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:46 smithi037 ceph-mon[29204]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:46 smithi069 ceph-mon[32534]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:46 smithi044 ceph-mon[32590]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:48.937 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:48 smithi069 ceph-mon[32534]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:48 smithi037 ceph-mon[29204]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:48 smithi044 ceph-mon[32590]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:50 smithi037 ceph-mon[29204]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:50 smithi069 ceph-mon[32534]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:50 smithi044 ceph-mon[32590]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:52 smithi037 ceph-mon[29204]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:52 smithi069 ceph-mon[32534]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:52 smithi044 ceph-mon[32590]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:54.942 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:54 smithi044 ceph-mon[32590]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:54 smithi037 ceph-mon[29204]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:54 smithi069 ceph-mon[32534]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:57.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:56 smithi037 ceph-mon[29204]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:56 smithi069 ceph-mon[32534]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:56 smithi044 ceph-mon[32590]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:30:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:30:58 smithi037 ceph-mon[29204]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:30:58 smithi069 ceph-mon[32534]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:30:59.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:30:58 smithi044 ceph-mon[32590]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:00 smithi037 ceph-mon[29204]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:00 smithi069 ceph-mon[32534]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:00 smithi044 ceph-mon[32590]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:02 smithi037 ceph-mon[29204]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:02 smithi069 ceph-mon[32534]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:03.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:02 smithi044 ceph-mon[32590]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:04 smithi037 ceph-mon[29204]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:04 smithi069 ceph-mon[32534]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:04 smithi044 ceph-mon[32590]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:06 smithi037 ceph-mon[29204]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:06 smithi069 ceph-mon[32534]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:06 smithi044 ceph-mon[32590]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:08 smithi037 ceph-mon[29204]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:08 smithi069 ceph-mon[32534]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:08 smithi044 ceph-mon[32590]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:10 smithi037 ceph-mon[29204]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:10 smithi069 ceph-mon[32534]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:10 smithi044 ceph-mon[32590]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:12 smithi037 ceph-mon[29204]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:12 smithi069 ceph-mon[32534]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:12 smithi044 ceph-mon[32590]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:31:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:14 smithi037 ceph-mon[29204]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:14 smithi069 ceph-mon[32534]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:15.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:14 smithi044 ceph-mon[32590]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:31:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:16 smithi037 ceph-mon[29204]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:31:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:16 smithi069 ceph-mon[32534]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:31:17.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:16 smithi044 ceph-mon[32590]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:31:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:18 smithi037 ceph-mon[29204]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:31:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:18 smithi069 ceph-mon[32534]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:31:19.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:18 smithi044 ceph-mon[32590]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:31:21.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:20 smithi037 ceph-mon[29204]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:20 smithi069 ceph-mon[32534]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:21.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:20 smithi044 ceph-mon[32590]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:31:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:31:22.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:31:22.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:31:22.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:31:22.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:31:22.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:31:22.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:31:22.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:31:22.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:31:22.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:31:22.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:31:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:22 smithi037 ceph-mon[29204]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:22 smithi069 ceph-mon[32534]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:22 smithi044 ceph-mon[32590]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:25.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:24 smithi037 ceph-mon[29204]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:24 smithi069 ceph-mon[32534]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:25.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:24 smithi044 ceph-mon[32590]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:26 smithi037 ceph-mon[29204]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:26 smithi069 ceph-mon[32534]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:26 smithi044 ceph-mon[32590]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:28 smithi037 ceph-mon[29204]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:28 smithi069 ceph-mon[32534]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:28 smithi044 ceph-mon[32590]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:30 smithi037 ceph-mon[29204]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:30 smithi069 ceph-mon[32534]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:31.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:30 smithi044 ceph-mon[32590]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:32 smithi037 ceph-mon[29204]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:32 smithi069 ceph-mon[32534]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:33.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:32 smithi044 ceph-mon[32590]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:34 smithi037 ceph-mon[29204]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:34 smithi069 ceph-mon[32534]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:34 smithi044 ceph-mon[32590]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:36 smithi037 ceph-mon[29204]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:36 smithi069 ceph-mon[32534]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:36 smithi044 ceph-mon[32590]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:38 smithi037 ceph-mon[29204]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:38 smithi069 ceph-mon[32534]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:38 smithi044 ceph-mon[32590]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:40 smithi037 ceph-mon[29204]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:40 smithi069 ceph-mon[32534]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:40 smithi044 ceph-mon[32590]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:42 smithi037 ceph-mon[29204]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:42 smithi069 ceph-mon[32534]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:42 smithi044 ceph-mon[32590]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:44 smithi037 ceph-mon[29204]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:44 smithi069 ceph-mon[32534]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:45.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:44 smithi044 ceph-mon[32590]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:46 smithi037 ceph-mon[29204]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:46 smithi069 ceph-mon[32534]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:47.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:46 smithi044 ceph-mon[32590]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:48 smithi037 ceph-mon[29204]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:48 smithi069 ceph-mon[32534]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:49.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:48 smithi044 ceph-mon[32590]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:50 smithi037 ceph-mon[29204]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:50 smithi069 ceph-mon[32534]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:50 smithi044 ceph-mon[32590]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:53.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:52 smithi037 ceph-mon[29204]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:52 smithi069 ceph-mon[32534]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:53.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:52 smithi044 ceph-mon[32590]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:54 smithi037 ceph-mon[29204]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:54 smithi069 ceph-mon[32534]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:54 smithi044 ceph-mon[32590]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:57.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:56 smithi037 ceph-mon[29204]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:56 smithi069 ceph-mon[32534]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:56 smithi044 ceph-mon[32590]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:31:59.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:31:58 smithi037 ceph-mon[29204]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:31:58 smithi069 ceph-mon[32534]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:31:59.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:31:58 smithi044 ceph-mon[32590]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:00 smithi037 ceph-mon[29204]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:00 smithi069 ceph-mon[32534]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:01.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:00 smithi044 ceph-mon[32590]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:02 smithi037 ceph-mon[29204]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:03.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:02 smithi069 ceph-mon[32534]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:03.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:02 smithi044 ceph-mon[32590]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:04 smithi037 ceph-mon[29204]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:04 smithi069 ceph-mon[32534]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:04 smithi044 ceph-mon[32590]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:06 smithi037 ceph-mon[29204]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:06 smithi069 ceph-mon[32534]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:06 smithi044 ceph-mon[32590]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:08 smithi037 ceph-mon[29204]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:08 smithi069 ceph-mon[32534]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:08 smithi044 ceph-mon[32590]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:10 smithi037 ceph-mon[29204]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:10 smithi069 ceph-mon[32534]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:10 smithi044 ceph-mon[32590]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:12 smithi037 ceph-mon[29204]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:12 smithi069 ceph-mon[32534]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:12 smithi044 ceph-mon[32590]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T13:32:15.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:14 smithi037 ceph-mon[29204]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:14 smithi069 ceph-mon[32534]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:14 smithi044 ceph-mon[32590]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T13:32:17.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:16 smithi037 ceph-mon[29204]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:32:17.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:16 smithi069 ceph-mon[32534]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:32:17.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:16 smithi044 ceph-mon[32590]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T13:32:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:18 smithi037 ceph-mon[29204]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:32:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:18 smithi069 ceph-mon[32534]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:32:19.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:18 smithi044 ceph-mon[32590]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T13:32:21.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:20 smithi037 ceph-mon[29204]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:20 smithi069 ceph-mon[32534]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:21.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:20 smithi044 ceph-mon[32590]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:22.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:21 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:22.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:21 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:22.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:21 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:32:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:22 smithi037 ceph-mon[29204]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:23.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:22 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:32:23.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:22 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:32:23.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:22 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:32:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:22 smithi069 ceph-mon[32534]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:22 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:32:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:22 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:32:23.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:22 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:32:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:22 smithi044 ceph-mon[32590]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:22 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:32:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:22 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:32:23.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:22 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:32:25.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:24 smithi037 ceph-mon[29204]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:24 smithi069 ceph-mon[32534]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:25.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:24 smithi044 ceph-mon[32590]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:26 smithi037 ceph-mon[29204]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:26 smithi069 ceph-mon[32534]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:27.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:26 smithi044 ceph-mon[32590]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:28 smithi037 ceph-mon[29204]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:28 smithi069 ceph-mon[32534]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:28 smithi044 ceph-mon[32590]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:30 smithi037 ceph-mon[29204]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:30 smithi069 ceph-mon[32534]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:31.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:30 smithi044 ceph-mon[32590]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:32 smithi037 ceph-mon[29204]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:32 smithi069 ceph-mon[32534]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:33.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:32 smithi044 ceph-mon[32590]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:34 smithi037 ceph-mon[29204]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:34 smithi069 ceph-mon[32534]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:35.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:34 smithi044 ceph-mon[32590]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:36 smithi037 ceph-mon[29204]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:36 smithi069 ceph-mon[32534]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:37.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:36 smithi044 ceph-mon[32590]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:38 smithi037 ceph-mon[29204]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:38 smithi069 ceph-mon[32534]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:39.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:38 smithi044 ceph-mon[32590]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:40 smithi037 ceph-mon[29204]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:40 smithi069 ceph-mon[32534]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:41.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:40 smithi044 ceph-mon[32590]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:42 smithi037 ceph-mon[29204]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:42 smithi069 ceph-mon[32534]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:43.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:42 smithi044 ceph-mon[32590]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:44 smithi037 ceph-mon[29204]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:44 smithi069 ceph-mon[32534]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:45.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:44 smithi044 ceph-mon[32590]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:46 smithi037 ceph-mon[29204]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:47.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:46 smithi069 ceph-mon[32534]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:46 smithi044 ceph-mon[32590]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:48 smithi037 ceph-mon[29204]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:48 smithi069 ceph-mon[32534]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:48 smithi044 ceph-mon[32590]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:50 smithi037 ceph-mon[29204]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:50 smithi069 ceph-mon[32534]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:51.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:50 smithi044 ceph-mon[32590]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:53.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:52 smithi037 ceph-mon[29204]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:52 smithi069 ceph-mon[32534]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:53.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:52 smithi044 ceph-mon[32590]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:54 smithi037 ceph-mon[29204]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:54 smithi069 ceph-mon[32534]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:55.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:54 smithi044 ceph-mon[32590]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:56 smithi069 ceph-mon[32534]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:57.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:56 smithi044 ceph-mon[32590]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:57.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:56 smithi037 ceph-mon[29204]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:32:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:32:58 smithi069 ceph-mon[32534]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:59.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:32:58 smithi044 ceph-mon[32590]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:32:59.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:32:58 smithi037 ceph-mon[29204]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:00 smithi069 ceph-mon[32534]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:01.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:00 smithi044 ceph-mon[32590]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:01.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:00 smithi037 ceph-mon[29204]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:02 smithi069 ceph-mon[32534]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:03.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:02 smithi044 ceph-mon[32590]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:03.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:02 smithi037 ceph-mon[29204]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:04 smithi069 ceph-mon[32534]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:33:05.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:04 smithi044 ceph-mon[32590]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:33:05.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:04 smithi037 ceph-mon[29204]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:33:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:06 smithi069 ceph-mon[32534]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:33:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:06 smithi044 ceph-mon[32590]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:33:07.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:06 smithi037 ceph-mon[29204]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:33:08.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:07 smithi069 ceph-mon[32534]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:33:08.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:07 smithi044 ceph-mon[32590]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:33:08.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:07 smithi037 ceph-mon[29204]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T13:33:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:08 smithi069 ceph-mon[32534]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:08 smithi044 ceph-mon[32590]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:09.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:08 smithi037 ceph-mon[29204]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:11.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:10 smithi044 ceph-mon[32590]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:11.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:10 smithi037 ceph-mon[29204]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:11.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:10 smithi069 ceph-mon[32534]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:13.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:12 smithi044 ceph-mon[32590]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:13.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:12 smithi037 ceph-mon[29204]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:13.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:12 smithi069 ceph-mon[32534]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:33:15.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:14 smithi037 ceph-mon[29204]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:33:15.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:14 smithi069 ceph-mon[32534]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:33:15.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:14 smithi044 ceph-mon[32590]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:33:16.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:16 smithi037 ceph-mon[29204]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:33:16.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:16 smithi037 ceph-mon[29204]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 241 B/s wr, 2 op/s 2024-04-25T13:33:16.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:16 smithi069 ceph-mon[32534]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:33:16.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:16 smithi069 ceph-mon[32534]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 241 B/s wr, 2 op/s 2024-04-25T13:33:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:16 smithi044 ceph-mon[32590]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T13:33:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:16 smithi044 ceph-mon[32590]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 241 B/s wr, 2 op/s 2024-04-25T13:33:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:18 smithi037 ceph-mon[29204]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:18.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:18 smithi069 ceph-mon[32534]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:18 smithi044 ceph-mon[32590]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:20 smithi037 ceph-mon[29204]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:20.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:20 smithi069 ceph-mon[32534]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:20 smithi044 ceph-mon[32590]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:22 smithi037 ceph-mon[29204]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:22 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:22 smithi069 ceph-mon[32534]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:22 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:22 smithi044 ceph-mon[32590]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:33:22.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:22 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:33:23.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:23 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:23.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:23 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:23.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:23 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:33:23.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:23 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:23.909 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:23 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:23.909 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:23 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:33:23.910 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:23 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:33:23.910 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:23 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:33:23.910 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:23 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:33:24.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:24 smithi037 ceph-mon[29204]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:24.790 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:24 smithi044 ceph-mon[32590]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:24.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:24 smithi069 ceph-mon[32534]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:26 smithi037 ceph-mon[29204]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:26 smithi069 ceph-mon[32534]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:26.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:26 smithi044 ceph-mon[32590]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:33:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:28 smithi037 ceph-mon[29204]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:33:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:28 smithi069 ceph-mon[32534]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:33:28.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:28 smithi044 ceph-mon[32590]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:33:30.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:30 smithi037 ceph-mon[29204]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:30.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:30 smithi069 ceph-mon[32534]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:30.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:30 smithi044 ceph-mon[32590]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:32.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:32 smithi037 ceph-mon[29204]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:32.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:32 smithi069 ceph-mon[32534]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:32.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:32 smithi044 ceph-mon[32590]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:34.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:34 smithi037 ceph-mon[29204]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:34.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:34 smithi069 ceph-mon[32534]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:34.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:34 smithi044 ceph-mon[32590]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:36 smithi037 ceph-mon[29204]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:36.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:36 smithi069 ceph-mon[32534]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:36.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:36 smithi044 ceph-mon[32590]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:38 smithi037 ceph-mon[29204]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:38.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:38 smithi069 ceph-mon[32534]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:38.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:38 smithi044 ceph-mon[32590]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:40 smithi037 ceph-mon[29204]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:40.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:40 smithi069 ceph-mon[32534]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:40.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:40 smithi044 ceph-mon[32590]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:42.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:42 smithi037 ceph-mon[29204]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:42.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:42 smithi069 ceph-mon[32534]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:42.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:42 smithi044 ceph-mon[32590]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:44 smithi037 ceph-mon[29204]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:44 smithi069 ceph-mon[32534]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:44.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:44 smithi044 ceph-mon[32590]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:46.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:46 smithi037 ceph-mon[29204]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:46.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:46 smithi069 ceph-mon[32534]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:46.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:46 smithi044 ceph-mon[32590]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:48.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:48 smithi037 ceph-mon[29204]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:48.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:48 smithi069 ceph-mon[32534]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:48.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:48 smithi044 ceph-mon[32590]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:50 smithi037 ceph-mon[29204]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:50 smithi069 ceph-mon[32534]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:50.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:50 smithi044 ceph-mon[32590]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:52.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:52 smithi037 ceph-mon[29204]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:52.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:52 smithi069 ceph-mon[32534]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:52.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:52 smithi044 ceph-mon[32590]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:54.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:54 smithi037 ceph-mon[29204]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:54.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:54 smithi069 ceph-mon[32534]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:54.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:54 smithi044 ceph-mon[32590]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:56 smithi037 ceph-mon[29204]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:56 smithi069 ceph-mon[32534]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:56.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:56 smithi044 ceph-mon[32590]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:33:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:33:58 smithi037 ceph-mon[29204]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:33:58 smithi069 ceph-mon[32534]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:33:58.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:33:58 smithi044 ceph-mon[32590]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:00 smithi037 ceph-mon[29204]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:00 smithi069 ceph-mon[32534]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:00 smithi044 ceph-mon[32590]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:02 smithi037 ceph-mon[29204]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:02 smithi069 ceph-mon[32534]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:02.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:02 smithi044 ceph-mon[32590]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:04.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:04 smithi037 ceph-mon[29204]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:04 smithi069 ceph-mon[32534]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:04.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:04 smithi044 ceph-mon[32590]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:06 smithi037 ceph-mon[29204]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:34:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:06 smithi069 ceph-mon[32534]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:34:06.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:06 smithi044 ceph-mon[32590]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:34:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:08 smithi037 ceph-mon[29204]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:08 smithi037 ceph-mon[29204]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:34:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:08 smithi069 ceph-mon[32534]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:08 smithi069 ceph-mon[32534]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:34:08.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:08 smithi044 ceph-mon[32590]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:34:08.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:08 smithi044 ceph-mon[32590]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T13:34:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:10 smithi037 ceph-mon[29204]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:10.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:10 smithi069 ceph-mon[32534]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:10 smithi044 ceph-mon[32590]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:12 smithi037 ceph-mon[29204]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:12 smithi069 ceph-mon[32534]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:12.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:12 smithi044 ceph-mon[32590]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:34:14.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:14 smithi037 ceph-mon[29204]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:34:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:14 smithi069 ceph-mon[32534]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:34:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:14 smithi044 ceph-mon[32590]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:34:16.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:16 smithi037 ceph-mon[29204]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:34:16.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:16 smithi037 ceph-mon[29204]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:34:16.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:16 smithi069 ceph-mon[32534]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:34:16.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:16 smithi069 ceph-mon[32534]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:34:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:16 smithi044 ceph-mon[32590]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T13:34:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:16 smithi044 ceph-mon[32590]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:34:18.853 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:18 smithi037 ceph-mon[29204]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:34:18.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:18 smithi069 ceph-mon[32534]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:34:18.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:18 smithi044 ceph-mon[32590]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:34:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:20 smithi037 ceph-mon[29204]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:20.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:20 smithi069 ceph-mon[32534]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:20.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:20 smithi044 ceph-mon[32590]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:22 smithi037 ceph-mon[29204]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:22 smithi069 ceph-mon[32534]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:22.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:22 smithi044 ceph-mon[32590]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:34:23.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:23 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:23.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:23 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:23.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:23 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:34:24.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:24 smithi037 ceph-mon[29204]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:24.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:24 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:24.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:24 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:24.753 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:24 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:34:24.791 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:24 smithi044 ceph-mon[32590]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:24.791 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:24 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:24.792 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:24 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:24.792 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:24 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:34:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:24 smithi069 ceph-mon[32534]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:24.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:24 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:34:24.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:24 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:34:24.886 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:24 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:34:26.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:26 smithi037 ceph-mon[29204]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:26.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:26 smithi069 ceph-mon[32534]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:26.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:26 smithi044 ceph-mon[32590]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:34:28.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:28 smithi037 ceph-mon[29204]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:34:28.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:28 smithi069 ceph-mon[32534]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:34:28.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:28 smithi044 ceph-mon[32590]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:34:30.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:30 smithi037 ceph-mon[29204]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:30.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:30 smithi069 ceph-mon[32534]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:30.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:30 smithi044 ceph-mon[32590]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:32.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:32 smithi037 ceph-mon[29204]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:32.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:32 smithi069 ceph-mon[32534]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:32.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:32 smithi044 ceph-mon[32590]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:34.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:34 smithi037 ceph-mon[29204]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:34.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:34 smithi069 ceph-mon[32534]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:34.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:34 smithi044 ceph-mon[32590]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:36.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:36 smithi037 ceph-mon[29204]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:36.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:36 smithi069 ceph-mon[32534]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:36.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:36 smithi044 ceph-mon[32590]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:38.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:38 smithi037 ceph-mon[29204]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:38.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:38 smithi069 ceph-mon[32534]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:38.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:38 smithi044 ceph-mon[32590]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:40 smithi037 ceph-mon[29204]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:40.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:40 smithi069 ceph-mon[32534]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:40.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:40 smithi044 ceph-mon[32590]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:42.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:42 smithi037 ceph-mon[29204]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:42.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:42 smithi069 ceph-mon[32534]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:42.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:42 smithi044 ceph-mon[32590]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:44.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:44 smithi037 ceph-mon[29204]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:44.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:44 smithi069 ceph-mon[32534]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:44.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:44 smithi044 ceph-mon[32590]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:46.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:46 smithi037 ceph-mon[29204]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:46.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:46 smithi069 ceph-mon[32534]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:46.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:46 smithi044 ceph-mon[32590]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:48.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:48 smithi037 ceph-mon[29204]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:48.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:48 smithi069 ceph-mon[32534]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:48.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:48 smithi044 ceph-mon[32590]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:50.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:50 smithi037 ceph-mon[29204]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:50.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:50 smithi069 ceph-mon[32534]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:50.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:50 smithi044 ceph-mon[32590]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:52.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:52 smithi037 ceph-mon[29204]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:52.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:52 smithi069 ceph-mon[32534]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:52.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:52 smithi044 ceph-mon[32590]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:54.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:54 smithi037 ceph-mon[29204]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:54.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:54 smithi069 ceph-mon[32534]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:54.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:54 smithi044 ceph-mon[32590]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:56.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:56 smithi037 ceph-mon[29204]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:56.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:56 smithi069 ceph-mon[32534]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:56.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:56 smithi044 ceph-mon[32590]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:34:58.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:34:58 smithi037 ceph-mon[29204]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:58.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:34:58 smithi069 ceph-mon[32534]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:34:58.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:34:58 smithi044 ceph-mon[32590]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:00.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:00 smithi037 ceph-mon[29204]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:00.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:00 smithi069 ceph-mon[32534]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:00.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:00 smithi044 ceph-mon[32590]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:02.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:02 smithi037 ceph-mon[29204]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:02.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:02 smithi069 ceph-mon[32534]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:02.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:02 smithi044 ceph-mon[32590]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:04.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:04 smithi037 ceph-mon[29204]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:04.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:04 smithi069 ceph-mon[32534]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:04.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:04 smithi044 ceph-mon[32590]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:06.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:06 smithi037 ceph-mon[29204]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:35:06.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:06 smithi069 ceph-mon[32534]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:35:06.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:06 smithi044 ceph-mon[32590]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:35:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:08 smithi037 ceph-mon[29204]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:08.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:08 smithi037 ceph-mon[29204]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:35:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:08 smithi069 ceph-mon[32534]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:08.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:08 smithi069 ceph-mon[32534]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:35:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:08 smithi044 ceph-mon[32590]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:35:08.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:08 smithi044 ceph-mon[32590]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T13:35:10.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:10 smithi037 ceph-mon[29204]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:10.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:10 smithi069 ceph-mon[32534]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:10.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:10 smithi044 ceph-mon[32590]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:12.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:12 smithi037 ceph-mon[29204]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:12.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:12 smithi069 ceph-mon[32534]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:12.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:12 smithi044 ceph-mon[32590]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:35:14.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:14 smithi037 ceph-mon[29204]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:35:14.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:14 smithi069 ceph-mon[32534]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:35:14.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:14 smithi044 ceph-mon[32590]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T13:35:16.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:16 smithi037 ceph-mon[29204]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:35:16.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:16 smithi037 ceph-mon[29204]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:35:16.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:16 smithi069 ceph-mon[32534]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:35:16.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:16 smithi069 ceph-mon[32534]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:35:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:16 smithi044 ceph-mon[32590]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T13:35:16.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:16 smithi044 ceph-mon[32590]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:35:18.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:18 smithi037 ceph-mon[29204]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:35:18.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:18 smithi069 ceph-mon[32534]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:35:18.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:18 smithi044 ceph-mon[32590]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 103 B/s wr, 2 op/s 2024-04-25T13:35:20.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:20 smithi037 ceph-mon[29204]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:20.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:20 smithi069 ceph-mon[32534]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:20.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:20 smithi044 ceph-mon[32590]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:22.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:22 smithi037 ceph-mon[29204]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:22.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:22 smithi069 ceph-mon[32534]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:22.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:22 smithi044 ceph-mon[32590]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T13:35:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:24 smithi037 ceph-mon[29204]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:24.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:24 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:24 smithi069 ceph-mon[32534]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:24.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:24 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:24.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:24 smithi044 ceph-mon[32590]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:24.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:24 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:35:25.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:25 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:25.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:25 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:25.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:25 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:35:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:25 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:25 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:25.885 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:25 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:35:25.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:25 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:35:25.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:25 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:35:25.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:25 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:35:27.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:26 smithi037 ceph-mon[29204]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:26 smithi069 ceph-mon[32534]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:27.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:26 smithi044 ceph-mon[32590]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T13:35:29.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:28 smithi037 ceph-mon[29204]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:35:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:28 smithi069 ceph-mon[32534]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:35:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:28 smithi044 ceph-mon[32590]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T13:35:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:30 smithi037 ceph-mon[29204]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:30 smithi069 ceph-mon[32534]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:31.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:30 smithi044 ceph-mon[32590]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:33.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:32 smithi037 ceph-mon[29204]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:33.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:32 smithi069 ceph-mon[32534]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:33.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:32 smithi044 ceph-mon[32590]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:35.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:34 smithi037 ceph-mon[29204]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:35.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:34 smithi069 ceph-mon[32534]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:34 smithi044 ceph-mon[32590]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:37.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:36 smithi037 ceph-mon[29204]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:37.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:36 smithi069 ceph-mon[32534]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:37.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:36 smithi044 ceph-mon[32590]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:39.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:38 smithi037 ceph-mon[29204]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:39.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:38 smithi069 ceph-mon[32534]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:39.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:38 smithi044 ceph-mon[32590]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:41.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:40 smithi037 ceph-mon[29204]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:41.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:40 smithi069 ceph-mon[32534]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:40 smithi044 ceph-mon[32590]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:43.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:42 smithi037 ceph-mon[29204]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:43.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:42 smithi069 ceph-mon[32534]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:43.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:42 smithi044 ceph-mon[32590]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:45.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:44 smithi037 ceph-mon[29204]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:45.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:44 smithi069 ceph-mon[32534]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:45.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:44 smithi044 ceph-mon[32590]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:47.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:46 smithi037 ceph-mon[29204]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:47.137 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:46 smithi069 ceph-mon[32534]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:47.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:46 smithi044 ceph-mon[32590]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:49.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:48 smithi037 ceph-mon[29204]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:49.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:48 smithi069 ceph-mon[32534]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:49.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:48 smithi044 ceph-mon[32590]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:51.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:50 smithi037 ceph-mon[29204]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:50 smithi069 ceph-mon[32534]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:51.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:50 smithi044 ceph-mon[32590]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:53.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:52 smithi037 ceph-mon[29204]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:53.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:52 smithi069 ceph-mon[32534]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:53.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:52 smithi044 ceph-mon[32590]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:55.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:54 smithi037 ceph-mon[29204]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:55.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:54 smithi069 ceph-mon[32534]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:55.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:54 smithi044 ceph-mon[32590]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:57.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:56 smithi037 ceph-mon[29204]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:57.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:56 smithi069 ceph-mon[32534]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:57.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:56 smithi044 ceph-mon[32590]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:35:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:35:58 smithi037 ceph-mon[29204]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:59.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:35:58 smithi069 ceph-mon[32534]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:35:59.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:35:58 smithi044 ceph-mon[32590]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:01.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:00 smithi037 ceph-mon[29204]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:01.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:00 smithi069 ceph-mon[32534]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:01.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:00 smithi044 ceph-mon[32590]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:03.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:02 smithi037 ceph-mon[29204]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:03.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:02 smithi069 ceph-mon[32534]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:03.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:02 smithi044 ceph-mon[32590]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:05.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:04 smithi037 ceph-mon[29204]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:36:05.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:04 smithi069 ceph-mon[32534]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:36:05.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:04 smithi044 ceph-mon[32590]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T13:36:07.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:06 smithi037 ceph-mon[29204]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:36:07.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:06 smithi069 ceph-mon[32534]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:36:07.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:06 smithi044 ceph-mon[32590]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T13:36:08.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:07 smithi037 ceph-mon[29204]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:36:08.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:07 smithi069 ceph-mon[32534]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:36:08.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:07 smithi044 ceph-mon[32590]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T13:36:09.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:08 smithi037 ceph-mon[29204]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:09.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:08 smithi069 ceph-mon[32534]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:09.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:08 smithi044 ceph-mon[32590]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:11.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:10 smithi037 ceph-mon[29204]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:11.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:10 smithi037 ceph-mon[29204]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:36:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:10 smithi069 ceph-mon[32534]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:11.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:10 smithi069 ceph-mon[32534]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:36:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:10 smithi044 ceph-mon[32590]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T13:36:11.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:10 smithi044 ceph-mon[32590]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T13:36:13.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:12 smithi037 ceph-mon[29204]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:36:13.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:12 smithi069 ceph-mon[32534]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:36:13.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:12 smithi044 ceph-mon[32590]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-25T13:36:14.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T13:36:14.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 -- 172.21.15.37:0/1836796932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401072a0 msgr2=0x7fd74010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 --2- 172.21.15.37:0/1836796932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401072a0 0x7fd74010ee70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd738009ca0 tx=0x7fd73802a8c0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 -- 172.21.15.37:0/1836796932 shutdown_connections 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 --2- 172.21.15.37:0/1836796932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401072a0 0x7fd74010ee70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 --2- 172.21.15.37:0/1836796932 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd740102b80 0x7fd740106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 --2- 172.21.15.37:0/1836796932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401021d0 0x7fd7401025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 -- 172.21.15.37:0/1836796932 >> 172.21.15.37:0/1836796932 conn(0x7fd7400fda10 msgr2=0x7fd7400ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 -- 172.21.15.37:0/1836796932 shutdown_connections 2024-04-25T13:36:14.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.140+0000 7fd744e62640 1 -- 172.21.15.37:0/1836796932 wait complete. 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 Processor -- start 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 -- start start 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401072a0 0x7fd740198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74019cf90 0x7fd74019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd740112210 con 0x7fd7401021d0 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd740112090 con 0x7fd7401072a0 2024-04-25T13:36:14.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.141+0000 7fd744e62640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd740112390 con 0x7fd74019cf90 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57082/0 (socket says 172.21.15.37:57082) 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 -- 172.21.15.37:0/2264855058 learned_addr learned my addr 172.21.15.37:0/2264855058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73dd9b640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401072a0 0x7fd740198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73ed9d640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74019cf90 0x7fd74019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 -- 172.21.15.37:0/2264855058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74019cf90 msgr2=0x7fd74019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74019cf90 0x7fd74019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 -- 172.21.15.37:0/2264855058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401072a0 msgr2=0x7fd740198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401072a0 0x7fd740198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 -- 172.21.15.37:0/2264855058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74019da80 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd73e59c640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c920 tx=0x7fd73000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730007c70 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd730007e10 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730018820 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74019dd70 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.142+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7401a55b0 con 0x7fd7401021d0 2024-04-25T13:36:14.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.144+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd740050530 con 0x7fd7401021d0 2024-04-25T13:36:14.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.145+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd73000f450 con 0x7fd7401021d0 2024-04-25T13:36:14.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.146+0000 7fd7377fe640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 0x7fd72c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.146+0000 7fd73dd9b640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 0x7fd72c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.146+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fd7300d08f0 con 0x7fd7401021d0 2024-04-25T13:36:14.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.146+0000 7fd73dd9b640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 0x7fd72c0a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7401088c0 tx=0x7fd72800b390 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.151+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd730014030 con 0x7fd7401021d0 2024-04-25T13:36:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.362+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd740102b80 con 0x7fd72c0a3d90 2024-04-25T13:36:14.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.370+0000 7fd7377fe640 1 -- 172.21.15.37:0/2264855058 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd740102b80 con 0x7fd72c0a3d90 2024-04-25T13:36:14.370 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T13:26:12", "first": "2024-04-25T13:27:00", "last": "2024-04-25T13:36:00", "last_pruned": "2024-04-25T13:36:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T13:36:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 msgr2=0x7fd72c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 0x7fd72c0a6250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd7401088c0 tx=0x7fd72800b390 comp rx=0 tx=0).stop 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 msgr2=0x7fd7401986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c920 tx=0x7fd73000cde0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 shutdown_connections 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fd72c0a3d90 0x7fd72c0a6250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd74019cf90 0x7fd74019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd7401072a0 0x7fd740198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 --2- 172.21.15.37:0/2264855058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7401021d0 0x7fd7401986d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 >> 172.21.15.37:0/2264855058 conn(0x7fd7400fda10 msgr2=0x7fd7400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:14.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 shutdown_connections 2024-04-25T13:36:14.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.372+0000 7fd744e62640 1 -- 172.21.15.37:0/2264855058 wait complete. 2024-04-25T13:36:14.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T13:36:14.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 -- 172.21.15.37:0/1416073805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec10c590 msgr2=0x7fccec10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 --2- 172.21.15.37:0/1416073805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec10c590 0x7fccec10e9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009ca0 tx=0x7fcce402a950 comp rx=0 tx=0).stop 2024-04-25T13:36:14.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 -- 172.21.15.37:0/1416073805 shutdown_connections 2024-04-25T13:36:14.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 --2- 172.21.15.37:0/1416073805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec10c590 0x7fccec10e9c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 --2- 172.21.15.37:0/1416073805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec103ae0 0x7fccec10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 --2- 172.21.15.37:0/1416073805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 -- 172.21.15.37:0/1416073805 >> 172.21.15.37:0/1416073805 conn(0x7fccec077f70 msgr2=0x7fccec078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 -- 172.21.15.37:0/1416073805 shutdown_connections 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.486+0000 7fccf094f640 1 -- 172.21.15.37:0/1416073805 wait complete. 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 Processor -- start 2024-04-25T13:36:14.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 -- start start 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 0x7fccec19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1121b0 con 0x7fccec103ae0 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fccec112030 con 0x7fccec10c590 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccf094f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccec112330 con 0x7fccec1031c0 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57100/0 (socket says 172.21.15.37:57100) 2024-04-25T13:36:14.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 -- 172.21.15.37:0/215815986 learned_addr learned my addr 172.21.15.37:0/215815986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccea59c640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccead9d640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 0x7fccec19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 -- 172.21.15.37:0/215815986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 msgr2=0x7fccec198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 -- 172.21.15.37:0/215815986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 msgr2=0x7fccec19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 0x7fccec19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fcce3fff640 1 -- 172.21.15.37:0/215815986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec19d6f0 con 0x7fccec103ae0 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccea59c640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:14.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.487+0000 7fccead9d640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 0x7fccec19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fcce3fff640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9f0 tx=0x7fccd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd400cd80 con 0x7fccec103ae0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec19d9e0 con 0x7fccec103ae0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd40108e0 con 0x7fccec103ae0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4018980 con 0x7fccec103ae0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.488+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1a5270 con 0x7fccec103ae0 2024-04-25T13:36:14.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.489+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec050530 con 0x7fccec103ae0 2024-04-25T13:36:14.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.490+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fccd4010430 con 0x7fccec103ae0 2024-04-25T13:36:14.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.490+0000 7fcce37fe640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 0x7fccd80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.491+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fccd40d0bf0 con 0x7fccec103ae0 2024-04-25T13:36:14.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.491+0000 7fccea59c640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 0x7fccd80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.491+0000 7fccea59c640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 0x7fccd80a61d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0058c0 tx=0x7fccdc005850 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.495+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccd409d260 con 0x7fccec103ae0 2024-04-25T13:36:14.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.705+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 --> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fccec1076e0 con 0x7fccd80a3d10 2024-04-25T13:36:14.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.712+0000 7fcce37fe640 1 -- 172.21.15.37:0/215815986 <== mgr.15207 v2:172.21.15.37:6800/2142450660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fccec1076e0 con 0x7fccd80a3d10 2024-04-25T13:36:14.712 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 msgr2=0x7fccd80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 0x7fccd80a61d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0058c0 tx=0x7fccdc005850 comp rx=0 tx=0).stop 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 msgr2=0x7fccec198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e9f0 tx=0x7fccd400eeb0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 shutdown_connections 2024-04-25T13:36:14.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fccd80a3d10 0x7fccd80a61d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fccec10c590 0x7fccec19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fccec103ae0 0x7fccec198c90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 --2- 172.21.15.37:0/215815986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccec1031c0 0x7fccec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 >> 172.21.15.37:0/215815986 conn(0x7fccec077f70 msgr2=0x7fccec0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 shutdown_connections 2024-04-25T13:36:14.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.714+0000 7fccf094f640 1 -- 172.21.15.37:0/215815986 wait complete. 2024-04-25T13:36:14.726 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T13:36:14.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1571342674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b0406a1b0 msgr2=0x7f4b04106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1571342674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b0406a1b0 0x7f4b04106c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009900 tx=0x7f4af402a480 comp rx=0 tx=0).stop 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1571342674 shutdown_connections 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1571342674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b041008a0 0x7f4b04100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1571342674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b0406a1b0 0x7f4b04106c50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1571342674 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 0x7f4b04106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1571342674 >> 172.21.15.37:0/1571342674 conn(0x7f4b04076130 msgr2=0x7f4b04076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1571342674 shutdown_connections 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.839+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1571342674 wait complete. 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 Processor -- start 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 -- start start 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 0x7f4b041974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0406a1b0 0x7f4b04197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04111010 con 0x7f4b041008a0 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04110e90 con 0x7f4b0406a1b0 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0c0e8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b04111190 con 0x7f4b04069dd0 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57122/0 (socket says 172.21.15.37:57122) 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 -- 172.21.15.37:0/1955330770 learned_addr learned my addr 172.21.15.37:0/1955330770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b09e84640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 0x7f4b041974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b09683640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0406a1b0 0x7f4b04197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 -- 172.21.15.37:0/1955330770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 msgr2=0x7f4b041974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 0x7f4b041974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 -- 172.21.15.37:0/1955330770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0406a1b0 msgr2=0x7f4b04197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0406a1b0 0x7f4b04197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.840+0000 7f4b0a685640 1 -- 172.21.15.37:0/1955330770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0419c4e0 con 0x7f4b041008a0 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4b0a685640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cce0 tx=0x7f4afc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00f450 con 0x7f4b041008a0 2024-04-25T13:36:14.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4afc00faa0 con 0x7f4b041008a0 2024-04-25T13:36:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc018b70 con 0x7f4b041008a0 2024-04-25T13:36:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0419c7d0 con 0x7f4b041008a0 2024-04-25T13:36:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.841+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b041a4060 con 0x7f4b041008a0 2024-04-25T13:36:14.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.842+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b04069250 con 0x7f4b041008a0 2024-04-25T13:36:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.843+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4afc018d10 con 0x7f4b041008a0 2024-04-25T13:36:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.844+0000 7f4afaffd640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 0x7f4af00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:14.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.844+0000 7f4b09e84640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 0x7f4af00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.844+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4afc0d0d00 con 0x7f4b041008a0 2024-04-25T13:36:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.844+0000 7f4b09e84640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 0x7f4af00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000bbe0 tx=0x7f4b0000b390 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:14.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:14.847+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4afc014030 con 0x7f4b041008a0 2024-04-25T13:36:15.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.035+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b041a4300 con 0x7f4b041008a0 2024-04-25T13:36:15.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.037+0000 7f4afaffd640 1 -- 172.21.15.37:0/1955330770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23072 (secure 0 0 0) 0x7f4afc09d400 con 0x7f4b041008a0 2024-04-25T13:36:15.037 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:15.038 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:36:10.291364+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1479124970":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/4004466629":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/774754895":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/683134189":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:36:15.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 msgr2=0x7f4af00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 0x7f4af00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000bbe0 tx=0x7f4b0000b390 comp rx=0 tx=0).stop 2024-04-25T13:36:15.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 msgr2=0x7f4b0419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00cce0 tx=0x7f4afc00a620 comp rx=0 tx=0).stop 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 shutdown_connections 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4af00a3d20 0x7f4af00a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b041008a0 0x7f4b0419bdb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b0406a1b0 0x7f4b04197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 --2- 172.21.15.37:0/1955330770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b04069dd0 0x7f4b041974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 >> 172.21.15.37:0/1955330770 conn(0x7f4b04076130 msgr2=0x7f4b041041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 shutdown_connections 2024-04-25T13:36:15.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.039+0000 7f4b0c0e8640 1 -- 172.21.15.37:0/1955330770 wait complete. 2024-04-25T13:36:15.048 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:14 smithi037 ceph-mon[29204]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:36:15.050 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:36:15.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:36:15.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:14 smithi069 ceph-mon[32534]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:36:15.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:14 smithi044 ceph-mon[32590]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 -- 172.21.15.37:0/3666780664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 msgr2=0x7f1018101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3666780664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018101560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1008009930 tx=0x7f100802a420 comp rx=0 tx=0).stop 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 -- 172.21.15.37:0/3666780664 shutdown_connections 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3666780664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 0x7f101810de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3666780664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 0x7f1018102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3666780664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018101560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 -- 172.21.15.37:0/3666780664 >> 172.21.15.37:0/3666780664 conn(0x7f1018077f70 msgr2=0x7f1018078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 -- 172.21.15.37:0/3666780664 shutdown_connections 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.155+0000 7f101e1b4640 1 -- 172.21.15.37:0/3666780664 wait complete. 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 Processor -- start 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 -- start start 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 0x7f1018198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 0x7f101819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10181111d0 con 0x7f1018101180 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1018111050 con 0x7f1018102a60 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f101e1b4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1018111350 con 0x7f1018103400 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f10177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f10177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57152/0 (socket says 172.21.15.37:57152) 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f10177fe640 1 -- 172.21.15.37:0/3663946503 learned_addr learned my addr 172.21.15.37:0/3663946503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.156+0000 7f100ffff640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 0x7f1018198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f1017fff640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 0x7f101819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 -- 172.21.15.37:0/3663946503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 msgr2=0x7f101819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 0x7f101819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 -- 172.21.15.37:0/3663946503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 msgr2=0x7f1018198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 0x7f1018198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 -- 172.21.15.37:0/3663946503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101819d670 con 0x7f1018101180 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10177fe640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1008031440 tx=0x7f1008003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100803f070 con 0x7f1018101180 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1008031d60 con 0x7f1018101180 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101819d900 con 0x7f1018101180 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008035030 con 0x7f1018101180 2024-04-25T13:36:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.157+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10181a51f0 con 0x7f1018101180 2024-04-25T13:36:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.158+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1018069250 con 0x7f1018101180 2024-04-25T13:36:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.159+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1008042070 con 0x7f1018101180 2024-04-25T13:36:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.160+0000 7f10157fa640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 0x7f10040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.160+0000 7f100ffff640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 0x7f10040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.160+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f10080f1720 con 0x7f1018101180 2024-04-25T13:36:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.161+0000 7f100ffff640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 0x7f10040a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10180785e0 tx=0x7f100000a2d0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.164+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f100803be20 con 0x7f1018101180 2024-04-25T13:36:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.351+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f101810a460 con 0x7f1018101180 2024-04-25T13:36:15.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.351+0000 7f10157fa640 1 -- 172.21.15.37:0/3663946503 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f10080bdd30 con 0x7f1018101180 2024-04-25T13:36:15.352 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:36:15.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 msgr2=0x7f10040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 0x7f10040a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f10180785e0 tx=0x7f100000a2d0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 msgr2=0x7f1018198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1008031440 tx=0x7f1008003d50 comp rx=0 tx=0).stop 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 shutdown_connections 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f10040a3da0 0x7f10040a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1018103400 0x7f101819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1018102a60 0x7f1018198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 --2- 172.21.15.37:0/3663946503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1018101180 0x7f1018198680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 >> 172.21.15.37:0/3663946503 conn(0x7f1018077f70 msgr2=0x7f101806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 shutdown_connections 2024-04-25T13:36:15.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.353+0000 7f101e1b4640 1 -- 172.21.15.37:0/3663946503 wait complete. 2024-04-25T13:36:15.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:36:15.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 -- 172.21.15.37:0/2189256765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d80101070 msgr2=0x7f4d80101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 --2- 172.21.15.37:0/2189256765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d80101070 0x7f4d80101450 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402a410 comp rx=0 tx=0).stop 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 -- 172.21.15.37:0/2189256765 shutdown_connections 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 --2- 172.21.15.37:0/2189256765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80106140 0x7f4d8010dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 --2- 172.21.15.37:0/2189256765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80101a20 0x7f4d80105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 --2- 172.21.15.37:0/2189256765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d80101070 0x7f4d80101450 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 -- 172.21.15.37:0/2189256765 >> 172.21.15.37:0/2189256765 conn(0x7f4d800fc870 msgr2=0x7f4d800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 -- 172.21.15.37:0/2189256765 shutdown_connections 2024-04-25T13:36:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 -- 172.21.15.37:0/2189256765 wait complete. 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.470+0000 7f4d87007640 1 Processor -- start 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 -- start start 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 0x7f4d80198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80106140 0x7f4d80198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801121f0 con 0x7f4d8019cff0 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80112070 con 0x7f4d80101070 2024-04-25T13:36:15.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d87007640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80112370 con 0x7f4d80106140 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57172/0 (socket says 172.21.15.37:57172) 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 -- 172.21.15.37:0/40083168 learned_addr learned my addr 172.21.15.37:0/40083168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d7ffff640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80106140 0x7f4d80198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d84da3640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 0x7f4d80198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 -- 172.21.15.37:0/40083168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80106140 msgr2=0x7f4d80198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80106140 0x7f4d80198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 -- 172.21.15.37:0/40083168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 msgr2=0x7f4d80198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 0x7f4d80198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 -- 172.21.15.37:0/40083168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8019dae0 con 0x7f4d8019cff0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d84da3640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 0x7f4d80198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d855a4640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800eaa0 tx=0x7f4d7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.471+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800cdb0 con 0x7f4d8019cff0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.472+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78005020 con 0x7f4d8019cff0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.472+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8019ddd0 con 0x7f4d8019cff0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.472+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780116d0 con 0x7f4d8019cff0 2024-04-25T13:36:15.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.472+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a55c0 con 0x7f4d8019cff0 2024-04-25T13:36:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.473+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d80102540 con 0x7f4d8019cff0 2024-04-25T13:36:15.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.474+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d780051d0 con 0x7f4d8019cff0 2024-04-25T13:36:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.474+0000 7f4d7dffb640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 0x7f4d700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.474+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4d780d0930 con 0x7f4d8019cff0 2024-04-25T13:36:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.474+0000 7f4d84da3640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 0x7f4d700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.475+0000 7f4d84da3640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 0x7f4d700a6260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000c00 tx=0x7f4d74005a20 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.479+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d78016030 con 0x7f4d8019cff0 2024-04-25T13:36:15.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.664+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d80199a80 con 0x7f4d8019cff0 2024-04-25T13:36:15.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.664+0000 7f4d7dffb640 1 -- 172.21.15.37:0/40083168 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f4d7809cfa0 con 0x7f4d8019cff0 2024-04-25T13:36:15.665 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:36:15.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 msgr2=0x7f4d700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 0x7f4d700a6260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d74000c00 tx=0x7f4d74005a20 comp rx=0 tx=0).stop 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 msgr2=0x7f4d8019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800eaa0 tx=0x7f4d7800ef60 comp rx=0 tx=0).stop 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 shutdown_connections 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4d700a3da0 0x7f4d700a6260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d8019cff0 0x7f4d8019d4a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80106140 0x7f4d80198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 --2- 172.21.15.37:0/40083168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d80101070 0x7f4d80198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 >> 172.21.15.37:0/40083168 conn(0x7f4d800fc870 msgr2=0x7f4d800fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 shutdown_connections 2024-04-25T13:36:15.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.666+0000 7f4d87007640 1 -- 172.21.15.37:0/40083168 wait complete. 2024-04-25T13:36:15.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:36:15.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 -- 172.21.15.37:0/3353770769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102b30 msgr2=0x7f5a74106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/3353770769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102b30 0x7f5a74106b70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009900 tx=0x7f5a6402a410 comp rx=0 tx=0).stop 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 -- 172.21.15.37:0/3353770769 shutdown_connections 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/3353770769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 0x7f5a7410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/3353770769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a74102b30 0x7f5a74106b70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/3353770769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 0x7f5a74102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 -- 172.21.15.37:0/3353770769 >> 172.21.15.37:0/3353770769 conn(0x7f5a740fda00 msgr2=0x7f5a740ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 -- 172.21.15.37:0/3353770769 shutdown_connections 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.782+0000 7f5a7bf20640 1 -- 172.21.15.37:0/3353770769 wait complete. 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 Processor -- start 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 -- start start 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 0x7f5a74198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 0x7f5a74198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74112170 con 0x7f5a7419cfd0 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74111ff0 con 0x7f5a74102180 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7bf20640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741122f0 con 0x7f5a74107250 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7a4bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7a4bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57184/0 (socket says 172.21.15.37:57184) 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a7a4bd640 1 -- 172.21.15.37:0/2560111506 learned_addr learned my addr 172.21.15.37:0/2560111506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a79cbc640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 0x7f5a74198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.783+0000 7f5a794bb640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 0x7f5a74198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 -- 172.21.15.37:0/2560111506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 msgr2=0x7f5a74198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 0x7f5a74198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 -- 172.21.15.37:0/2560111506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 msgr2=0x7f5a74198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 0x7f5a74198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 -- 172.21.15.37:0/2560111506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7419dac0 con 0x7f5a7419cfd0 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7a4bd640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00cce0 tx=0x7f5a6c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00f450 con 0x7f5a7419cfd0 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a6c00faa0 con 0x7f5a7419cfd0 2024-04-25T13:36:15.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c018b70 con 0x7f5a7419cfd0 2024-04-25T13:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7419ddb0 con 0x7f5a7419cfd0 2024-04-25T13:36:15.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.784+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a5570 con 0x7f5a7419cfd0 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.785+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74069250 con 0x7f5a7419cfd0 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.786+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a6c018d10 con 0x7f5a7419cfd0 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.787+0000 7f5a6affd640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 0x7f5a600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.787+0000 7f5a79cbc640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 0x7f5a600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.787+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f5a6c0d0b10 con 0x7f5a7419cfd0 2024-04-25T13:36:15.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.787+0000 7f5a79cbc640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 0x7f5a600a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009bd0 tx=0x7f5a70005f90 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:15.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.792+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a6c014030 con 0x7f5a7419cfd0 2024-04-25T13:36:15.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.981+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a7410a4e0 con 0x7f5a7419cfd0 2024-04-25T13:36:15.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.981+0000 7f5a6affd640 1 -- 172.21.15.37:0/2560111506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f5a6c09d180 con 0x7f5a7419cfd0 2024-04-25T13:36:15.981 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:36:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 msgr2=0x7f5a600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 0x7f5a600a6260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009bd0 tx=0x7f5a70005f90 comp rx=0 tx=0).stop 2024-04-25T13:36:15.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 msgr2=0x7f5a7419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00cce0 tx=0x7f5a6c00a620 comp rx=0 tx=0).stop 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 shutdown_connections 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5a600a3da0 0x7f5a600a6260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a7419cfd0 0x7f5a7419d480 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a74107250 0x7f5a74198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 --2- 172.21.15.37:0/2560111506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a74102180 0x7f5a74198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 >> 172.21.15.37:0/2560111506 conn(0x7f5a740fda00 msgr2=0x7f5a740fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:15.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.983+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 shutdown_connections 2024-04-25T13:36:15.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:15.984+0000 7f5a7bf20640 1 -- 172.21.15.37:0/2560111506 wait complete. 2024-04-25T13:36:15.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:15 smithi037 ceph-mon[29204]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:15.993 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:15 smithi037 ceph-mon[29204]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:15.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1955330770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:15.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3663946503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:15.994 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:15 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/40083168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:15.995 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T13:36:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 -- 172.21.15.37:0/3100690581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42001021a0 msgr2=0x7f4200102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 --2- 172.21.15.37:0/3100690581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42001021a0 0x7f4200102580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009900 tx=0x7f41e802a410 comp rx=0 tx=0).stop 2024-04-25T13:36:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 -- 172.21.15.37:0/3100690581 shutdown_connections 2024-04-25T13:36:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 --2- 172.21.15.37:0/3100690581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4200107270 0x7f420010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:16.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 --2- 172.21.15.37:0/3100690581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4200102b50 0x7f4200106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 --2- 172.21.15.37:0/3100690581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42001021a0 0x7f4200102580 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.096+0000 7f42064e8640 1 -- 172.21.15.37:0/3100690581 >> 172.21.15.37:0/3100690581 conn(0x7f42000fda40 msgr2=0x7f42000ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- 172.21.15.37:0/3100690581 shutdown_connections 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- 172.21.15.37:0/3100690581 wait complete. 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 Processor -- start 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- start start 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42001021a0 0x7f42001987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:16.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f420019d020 0x7f420019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42001121d0 con 0x7f4200107270 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112050 con 0x7f420019d020 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.097+0000 7f42064e8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112350 con 0x7f42001021a0 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57196/0 (socket says 172.21.15.37:57196) 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 -- 172.21.15.37:0/942245391 learned_addr learned my addr 172.21.15.37:0/942245391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41fffff640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42001021a0 0x7f42001987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f4204a85640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f420019d020 0x7f420019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 -- 172.21.15.37:0/942245391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42001021a0 msgr2=0x7f42001987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42001021a0 0x7f42001987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 -- 172.21.15.37:0/942245391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f420019d020 msgr2=0x7f420019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f420019d020 0x7f420019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 -- 172.21.15.37:0/942245391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420019db10 con 0x7f4200107270 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41ff7fe640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f000d740 tx=0x7f41f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0017070 con 0x7f4200107270 2024-04-25T13:36:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420019de00 con 0x7f4200107270 2024-04-25T13:36:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f000de50 con 0x7f4200107270 2024-04-25T13:36:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004ce0 con 0x7f4200107270 2024-04-25T13:36:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.098+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a55f0 con 0x7f4200107270 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.099+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200069250 con 0x7f4200107270 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.100+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41f0004e80 con 0x7f4200107270 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.101+0000 7f41fd7fa640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.101+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f41f00d02f0 con 0x7f4200107270 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.101+0000 7f41fffff640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.102+0000 7f41fffff640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41e8031440 tx=0x7f41e80095f0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.106+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41f009c900 con 0x7f4200107270 2024-04-25T13:36:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:15 smithi069 ceph-mon[32534]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:15 smithi069 ceph-mon[32534]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:15 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1955330770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:16.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:15 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3663946503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:16.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:15 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/40083168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:16.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:15 smithi044 ceph-mon[32590]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:16.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:15 smithi044 ceph-mon[32590]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T13:36:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:15 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1955330770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:15 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3663946503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:16.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:15 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/40083168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:16.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.340+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4200102b50 con 0x7f4200107270 2024-04-25T13:36:16.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.349+0000 7f41fd7fa640 1 -- 172.21.15.37:0/942245391 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f41f009c200 con 0x7f4200107270 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 15207, 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-04-25T13:36:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2142450660 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T13:36:16.361 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2142450660 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/2142450660", 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-04-25T13:26:11.228431+0000", 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-04-25T13:36:16.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24866, 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.363 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.366 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.375 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T13:36:16.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T13:36:16.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T13:36:16.387 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T13:36:16.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:16.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T13:36:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:36:16.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.399 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.402 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T13:36:16.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T13:36:16.412 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T13:36:16.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:36:16.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:36:16.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:36:16.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.420 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T13:36:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:36:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:36:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T13:36:16.432 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T13:36:16.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:36:16.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T13:36:16.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:16.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T13:36:16.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:36:16.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.448 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:36:16.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:36:16.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T13:36:16.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T13:36:16.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:36:16.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T13:36:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T13:36:16.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T13:36:16.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:16.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.480 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T13:36:16.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.485 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.491 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:36:16.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:36:16.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T13:36:16.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:36:16.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T13:36:16.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:36:16.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T13:36:16.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:36:16.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.517 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T13:36:16.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T13:36:16.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.530 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.536 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.546 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:36:16.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:36:16.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:36:16.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.567 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:36:16.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.575 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:36:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:36:16.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:36:16.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:36:16.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:36:16.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:36:16.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T13:36:16.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.629 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.633 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.636 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:16.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.641 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.655 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.659 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T13:36:16.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.665 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.668 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T13:36:16.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.675 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.679 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:16.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.687 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.698 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T13:36:16.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T13:36:16.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:16.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:16.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.707 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.710 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.712 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.716 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.728 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T13:36:16.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T13:36:16.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:36:16.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.737 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:36:16.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T13:36:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T13:36:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T13:36:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T13:36:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.844 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.852 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.855 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T13:36:16.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:16.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.870 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T13:36:16.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.887 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:16.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.898 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.904 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.909 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.913 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T13:36:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.920 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.924 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T13:36:16.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T13:36:16.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.936 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.954 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.955 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.958 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.964 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.969 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.973 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.976 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.977 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.984 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:16.985 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:16.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:16.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T13:36:16.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T13:36:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T13:36:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T13:36:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T13:36:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T13:36:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24910, 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T13:36:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T13:36:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T13:36:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T13:36:17.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T13:36:17.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:17.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.103 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:17.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T13:36:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:36:17.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T13:36:17.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:36:17.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.121 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T13:36:17.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T13:36:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:36:17.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T13:36:17.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T13:36:17.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T13:36:17.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T13:36:17.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:36:17.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:36:17.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:36:17.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T13:36:17.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T13:36:17.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:36:17.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.154 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.157 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T13:36:17.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T13:36:17.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:36:17.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.172 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T13:36:17.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.187 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:36:17.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:36:17.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T13:36:17.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T13:36:17.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T13:36:17.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:36:17.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.205 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T13:36:17.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:36:17.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T13:36:17.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:36:17.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T13:36:17.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.223 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.225 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.228 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T13:36:17.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:36:17.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.234 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:36:17.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.238 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.243 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:36:17.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.254 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:36:17.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:36:17.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:36:17.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.268 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:36:17.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:36:17.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:36:17.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:36:17.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.286 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.293 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:36:17.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.297 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:36:17.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.303 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:36:17.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:36:17.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T13:36:17.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:36:17.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.323 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.327 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:17.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T13:36:17.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:17.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T13:36:17.340 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T13:36:17.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T13:36:17.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.348 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.352 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.353 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T13:36:17.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:36:17.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T13:36:17.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:36:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T13:36:17.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.367 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T13:36:17.381 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.383 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T13:36:17.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.389 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T13:36:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T13:36:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T13:36:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T13:36:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:36:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T13:36:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:36:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T13:36:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T13:36:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T13:36:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:36:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T13:36:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T13:36:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.521 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.522 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:36:17.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.528 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:17.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.533 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.537 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.544 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T13:36:17.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:36:17.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.549 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.555 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T13:36:17.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T13:36:17.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:17.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.570 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.574 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:36:17.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T13:36:17.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.587 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.591 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.594 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:17.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.600 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.604 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T13:36:17.607 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.609 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.619 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.623 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T13:36:17.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T13:36:17.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.634 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T13:36:17.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.639 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T13:36:17.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.644 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.651 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.656 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.659 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.661 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.665 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T13:36:17.668 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.674 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.677 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.679 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.682 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.686 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T13:36:17.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T13:36:17.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T13:36:17.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T13:36:17.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.713 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.717 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T13:36:17.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T13:36:17.725 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.727 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.731 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.732 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.736 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.740 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T13:36:17.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T13:36:17.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.757 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.761 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T13:36:17.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T13:36:17.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.769 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-04-25T13:36:17.770 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:17.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.774 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.775 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T13:36:17.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T13:36:17.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T13:36:17.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T13:36:17.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T13:36:17.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.795 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T13:36:17.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T13:36:17.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T13:36:17.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T13:36:17.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T13:36:17.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T13:36:17.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.810 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:17.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T13:36:17.821 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:17.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T13:36:17.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T13:36:17.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T13:36:17.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T13:36:17.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T13:36:17.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T13:36:17.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.853 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T13:36:17.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.858 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T13:36:17.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T13:36:17.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.868 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T13:36:17.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T13:36:17.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T13:36:17.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.881 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T13:36:17.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.886 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T13:36:17.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.898 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.901 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T13:36:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T13:36:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T13:36:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T13:36:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T13:36:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T13:36:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T13:36:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T13:36:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T13:36:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T13:36:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T13:36:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T13:36:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T13:36:18.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T13:36:18.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.017 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T13:36:18.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T13:36:18.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.020 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:18.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.025 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T13:36:18.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.030 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.035 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.039 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.040 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.050 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T13:36:18.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.064 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.065 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T13:36:18.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T13:36:18.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T13:36:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:18.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T13:36:18.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.081 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.087 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:18.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T13:36:18.095 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T13:36:18.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T13:36:18.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T13:36:18.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T13:36:18.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.103 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.112 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T13:36:18.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T13:36:18.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.122 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.125 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.126 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T13:36:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T13:36:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T13:36:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T13:36:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T13:36:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T13:36:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T13:36:18.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T13:36:18.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.285 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.287 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.291 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T13:36:18.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.299 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T13:36:18.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.304 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.308 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.309 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T13:36:18.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.315 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.320 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.324 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.330 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.335 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T13:36:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T13:36:18.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T13:36:18.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T13:36:18.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.373 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.383 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.388 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.392 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.395 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.396 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.401 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.402 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.406 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.409 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.413 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T13:36:18.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T13:36:18.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T13:36:18.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T13:36:18.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T13:36:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.431 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.440 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T13:36:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T13:36:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T13:36:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T13:36:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T13:36:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T13:36:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T13:36:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-04-25T13:36:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T13:36:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T13:36:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T13:36:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T13:36:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T13:36:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 92, 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3348268312 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3255327988 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T13:36:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 161036403 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1513232598 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4194091091 2024-04-25T13:36:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T13:36:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1802707903 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T13:36:18.507 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:36:18.508 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:16 smithi044 ceph-mon[32590]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 237 B/s wr, 2 op/s 2024-04-25T13:36:18.508 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:16 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2560111506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:18.509 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:16 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/942245391' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:36:18.509 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:16 smithi069 ceph-mon[32534]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 237 B/s wr, 2 op/s 2024-04-25T13:36:18.509 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:16 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2560111506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:18.509 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:16 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/942245391' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 msgr2=0x7f41ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41e8031440 tx=0x7f41e80095f0 comp rx=0 tx=0).stop 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 msgr2=0x7f4200198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f000d740 tx=0x7f41f000dc00 comp rx=0 tx=0).stop 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 shutdown_connections 2024-04-25T13:36:18.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f41ec0a3d20 0x7f41ec0a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f420019d020 0x7f420019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4200107270 0x7f4200198cf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 --2- 172.21.15.37:0/942245391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42001021a0 0x7f42001987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 >> 172.21.15.37:0/942245391 conn(0x7f42000fda40 msgr2=0x7f42000fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 shutdown_connections 2024-04-25T13:36:18.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:16.354+0000 7f42064e8640 1 -- 172.21.15.37:0/942245391 wait complete. 2024-04-25T13:36:18.520 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T13:36:18.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T13:36:18.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:16 smithi037 ceph-mon[29204]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 237 B/s wr, 2 op/s 2024-04-25T13:36:18.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:16 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2560111506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:18.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:16 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/942245391' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T13:36:18.782 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:19.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:18 smithi037 ceph-mon[29204]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:19.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:18 smithi069 ceph-mon[32534]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:19.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:18 smithi044 ceph-mon[32590]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:20.654 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:20.655 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001459640, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000297083, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.656 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:20.657 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 153, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 183008, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015080320, 2024-04-25T13:36:20.658 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009980114, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002418433, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 151, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014241979, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000094317 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:20.659 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 183008 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 210, 2024-04-25T13:36:20.660 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 199, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1586384, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 122144, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.043165761, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013968785, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.033238921, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004604688, 2024-04-25T13:36:20.661 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 198, 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.010955824, 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000055332 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:20.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1586384, 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 122144 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T13:36:20.663 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:20.664 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:20.665 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:20.666 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:20.667 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:20.668 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:20.669 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:20.670 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 212, 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1556800, 2024-04-25T13:36:20.673 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 212, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 212, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1556800, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:20.674 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:20.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:20.675 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:36:21.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:20 smithi037 ceph-mon[29204]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:21.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:20 smithi069 ceph-mon[32534]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:21.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:20 smithi044 ceph-mon[32590]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T13:36:21.527 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T13:36:21.527 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:36:21.527 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T13:36:21.527 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T13:36:21.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T13:36:21.790 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:23.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:22 smithi037 ceph-mon[29204]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 4 op/s 2024-04-25T13:36:23.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:22 smithi069 ceph-mon[32534]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 4 op/s 2024-04-25T13:36:23.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:22 smithi044 ceph-mon[32590]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 92 B/s wr, 4 op/s 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:23.627 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001202448, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000156748, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:23.628 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:23.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 158, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 183888, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.014979682, 2024-04-25T13:36:23.630 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009952613, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002247596, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 156, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011736211, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000075232 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:23.631 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 183888 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 215, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 204, 2024-04-25T13:36:23.632 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1587744, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 125344, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.043548130, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.014031761, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.034465286, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004410831, 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:23.633 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 203, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011673904, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000057506 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.634 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1587744, 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 125344 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:23.635 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:23.636 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:23.637 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:23.638 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-04-25T13:36:23.639 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:23.640 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:23.641 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 217, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1557450, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:23.642 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 217, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 217, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1557450, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:23.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:23.644 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:36:24.338 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T13:36:24.338 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:36:24.338 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T13:36:24.338 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T13:36:24.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T13:36:24.592 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:25.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:24 smithi037 ceph-mon[29204]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:36:25.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:24 smithi069 ceph-mon[32534]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:36:25.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:24 smithi044 ceph-mon[32590]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T13:36:25.921 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:25 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:26.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:25 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:26.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:25 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T13:36:26.399 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:36:26.399 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:26.399 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 212, 2024-04-25T13:36:26.399 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 201, 2024-04-25T13:36:26.399 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1586928, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 123424, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.042975202, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013812463, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.032256424, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004467859, 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:26.400 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 200, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011591224, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000057956 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.401 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1586928, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 123424 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 155, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 183360, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:26.402 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015294223, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.009928327, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002562616, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 153, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017143839, 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000112051 2024-04-25T13:36:26.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 183360 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.404 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000368429, 2024-04-25T13:36:26.405 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000279687, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:26.406 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T13:36:26.407 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:26.408 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:26.409 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:26.410 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:26.411 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T13:36:26.412 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:26.413 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:26.414 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 214, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1557060, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 214, 2024-04-25T13:36:26.415 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 214, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1557060, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:26.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:26.417 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:36:27.049 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T13:36:27.049 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:36:27.049 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T13:36:27.049 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T13:36:27.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T13:36:27.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:26 smithi037 ceph-mon[29204]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:36:27.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:26 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:26 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:26 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.072 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:26 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:26 smithi069 ceph-mon[32534]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:36:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:26 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:26 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:26 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:26 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:26 smithi044 ceph-mon[32590]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T13:36:27.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:26 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:26 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:26 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:26 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:27.844 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T13:36:28.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.084 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:27 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:27 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:27 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:28.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:28 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:36:28.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:28 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:36:28.923 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:28 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:36:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:28 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:36:28.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:28 smithi037 ceph-mon[29204]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:36:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:28 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:36:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:28 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:36:29.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:28 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:36:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:28 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:36:29.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:28 smithi069 ceph-mon[32534]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:36:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:28 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi069 to 176.1M 2024-04-25T13:36:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:28 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi069 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-04-25T13:36:29.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:28 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi044 to 177.2M 2024-04-25T13:36:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:28 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi044 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-25T13:36:29.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:28 smithi044 ceph-mon[32590]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T13:36:29.532 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T13:36:29.532 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:29.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 218, 2024-04-25T13:36:29.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 207, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1588560, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 127264, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.046879038, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013983465, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.036706923, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.004495693, 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:29.533 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 206, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.012140005, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000058932 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.534 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1588560, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 127264 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 158, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 183792, 2024-04-25T13:36:29.535 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.014958044, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010303792, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001762893, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 155, 2024-04-25T13:36:29.536 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.011504164, 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000074220 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:29.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 183792 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-25T13:36:29.538 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000742287, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000045924, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000673691, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000837427, 2024-04-25T13:36:29.539 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000279142 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:36:29.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:29.541 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:29.542 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:29.543 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:29.544 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:29.545 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:29.546 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T13:36:29.547 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 220, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1557840, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:29.548 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 220, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 220, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1557840, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T13:36:29.549 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:29.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T13:36:29.550 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T13:36:30.228 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T13:36:30.228 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:36:30.228 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T13:36:30.228 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T13:36:30.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T13:36:30.489 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:30.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:30 smithi044 ceph-mon[32590]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:30.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:30 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:30.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:30 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:30 smithi037 ceph-mon[29204]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:30 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:31.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:30 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:30 smithi069 ceph-mon[32534]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:30 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:31.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:30 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:31.889 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:31.890 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:31.891 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:31 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:32.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:36:32.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:36:32.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:32.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:32.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:31 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:32.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T13:36:32.136 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:31 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' 2024-04-25T13:36:32.231 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-25T13:36:32.231 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1161, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 606, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1972062, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 293698, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 9, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.159037942, 2024-04-25T13:36:32.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.039192652, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.079892678, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.047513606, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 605, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.223472698, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000369376 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:32.233 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 820, 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000172717, 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000210 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1919920, 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 220768 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:32.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 385, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 238, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 135663, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 199281, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 10, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.038700163, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.007661782, 2024-04-25T13:36:32.235 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.012044047, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.012623254, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 235, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.180222874, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000766905 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 206, 2024-04-25T13:36:32.236 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000035573, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000172 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 86448, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 133616 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1022, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 486, 2024-04-25T13:36:32.237 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 382719, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 576314, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 13, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.138932006, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.035368632, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.045817507, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.053343498, 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:32.238 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 485, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.260096815, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000536282 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 867, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000163722, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000188 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.239 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 56512, 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 342656 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-25T13:36:32.240 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 209, 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.060971161, 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000291728 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-25T13:36:32.241 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000007464, 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000003732 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 152, 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 152, 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 152, 2024-04-25T13:36:32.242 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.122285567, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000804510 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 12, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-04-25T13:36:32.243 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 25, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 2, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 23, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 22, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 2, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 2, 2024-04-25T13:36:32.244 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 2, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 193, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 185, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-04-25T13:36:32.245 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-04-25T13:36:32.246 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 4, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 234, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 6, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:36:32.247 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 5, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 14, 2024-04-25T13:36:32.248 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 2, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 17 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-04-25T13:36:32.249 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-04-25T13:36:32.250 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:36:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 101, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-04-25T13:36:32.252 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 101, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-04-25T13:36:32.253 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4283532, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 81, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.154580170, 2024-04-25T13:36:32.254 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001908397 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 23, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 23, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 0, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 18, 2024-04-25T13:36:32.255 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 18, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 25, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 25, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 0, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 2, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 17, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 15, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 53920, 2024-04-25T13:36:32.256 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 231800 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 174, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 152, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 31, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-04-25T13:36:32.257 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4, 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.003311726, 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000827931 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 103, 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.022824099, 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000221593 2024-04-25T13:36:32.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.259 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4, 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001099166, 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000274791 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.260 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-04-25T13:36:32.261 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.262 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 10, 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001709944, 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000170994 2024-04-25T13:36:32.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.264 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-04-25T13:36:32.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6, 2024-04-25T13:36:32.264 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.007071159, 2024-04-25T13:36:32.264 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001178526 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-04-25T13:36:32.265 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 10, 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.046677052, 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004667705 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000233055, 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000233055 2024-04-25T13:36:32.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.002636531, 2024-04-25T13:36:32.267 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001318265 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-04-25T13:36:32.268 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.016395170, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004098792 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.270 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000956626, 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000956626 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-04-25T13:36:32.271 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-04-25T13:36:32.272 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 7, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.019371039, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002767291 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.273 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 5, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 3, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 27, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 13, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 596, 2024-04-25T13:36:32.276 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:32.277 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:32.278 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:32.279 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-25T13:36:32.280 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 342459, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 133, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 704, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 8, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-25T13:36:32.281 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8224, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 171, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:32.282 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 93898, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 539, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-25T13:36:32.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 642, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 103338, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 32, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 642, 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-04-25T13:36:32.284 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 642, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 14.172773734, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.022075971 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 642, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 675 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.285 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 642, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 0, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 642, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 11, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 0, 2024-04-25T13:36:32.286 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 81, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 21, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 506, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-04-25T13:36:32.287 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 22, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-04-25T13:36:32.288 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-04-25T13:36:32.289 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-04-25T13:36:32.290 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 100, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 12, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 12, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 12, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-04-25T13:36:32.291 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 30, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 4 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-04-25T13:36:32.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-04-25T13:36:32.293 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2564, 2024-04-25T13:36:32.294 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 2196925, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2564, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2564, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2196925, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.295 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-25T13:36:32.296 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 642, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 103338, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 642, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 103338, 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:32.297 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:32.298 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 642, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 642, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 642, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 642, 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:32.299 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:32.300 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:32.301 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55bce5e94460": { 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-04-25T13:36:32.302 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 101, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 89228, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 101, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 81, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 89228, 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:32.303 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.304 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:32.304 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-25T13:36:32.976 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T13:36:32.976 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T13:36:32.977 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T13:36:32.977 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T13:36:32.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T13:36:33.227 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:33.347 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:32 smithi044 ceph-mon[32590]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:33.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:32 smithi037 ceph-mon[29204]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:33.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:32 smithi069 ceph-mon[32534]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:34.861 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 218, 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 207, 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1588560, 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 127264, 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:34.862 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.046118536, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.013594611, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.034255962, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.005564108, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 206, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.012177082, 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000059112 2024-04-25T13:36:34.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1588560, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 127264 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:34.864 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 160, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 184240, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.012331578, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.008169912, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001865476, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:34.865 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 158, 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.018174502, 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000115028 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:34.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 184240 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:34.867 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.000424904, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000368330, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:34.868 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.869 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:34.870 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:34.871 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:34.872 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:34.873 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 149057, 2024-04-25T13:36:34.874 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 32, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:34.875 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:34.876 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:34.877 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 220, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1557840, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 220, 2024-04-25T13:36:34.878 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 220, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1557840, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:34.879 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:34.880 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-25T13:36:35.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:34 smithi044 ceph-mon[32590]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:35.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:34 smithi037 ceph-mon[29204]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:35.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:34 smithi069 ceph-mon[32534]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:35.560 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T13:36:35.560 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T13:36:35.560 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T13:36:35.560 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T13:36:35.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T13:36:35.813 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:37.059 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:36 smithi044 ceph-mon[32590]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:37.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:36 smithi037 ceph-mon[29204]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:37.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:36 smithi069 ceph-mon[32534]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:37.660 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-25T13:36:37.660 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:37.660 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4, 2024-04-25T13:36:37.660 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 448, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001719422, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000039109, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000648012, 2024-04-25T13:36:37.661 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3, 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000718588, 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000239529 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:37.662 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 2, 2024-04-25T13:36:37.663 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 157, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 183616, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.011952992, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.007905035, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001533895, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:37.664 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 154, 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.018784349, 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000121976 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:37.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183616 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 216, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 205, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1588016, 2024-04-25T13:36:37.666 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 125984, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.044997375, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.013883939, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.033031845, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.005577187, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 204, 2024-04-25T13:36:37.667 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.011037407, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000054104 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1588016, 2024-04-25T13:36:37.668 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:37.669 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:37.670 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:37.671 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:37.672 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:37.673 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:37.674 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:37.675 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-25T13:36:37.676 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 218, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1557580, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 218, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 218, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1557580, 2024-04-25T13:36:37.677 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:37.678 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-25T13:36:38.352 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-25T13:36:38.352 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-25T13:36:38.352 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-25T13:36:38.352 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T13:36:38.353 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T13:36:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T13:36:38.978 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:38 smithi044 ceph-mon[32590]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:39.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:38 smithi037 ceph-mon[29204]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:39.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:38 smithi069 ceph-mon[32534]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:40.226 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-25T13:36:40.226 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 221, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 210, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1589376, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 129184, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.050489438, 2024-04-25T13:36:40.227 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.014551264, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.038832543, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.006038777, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 209, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.012076333, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000057781 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:40.228 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1589376, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 129184 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 163, 2024-04-25T13:36:40.229 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3264, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 184768, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.012837873, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.008285042, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.002036301, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:40.230 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 161, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013085001, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000081273 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.231 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 184768 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:40.232 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.000321997, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000261921, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:40.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.234 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:40.235 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:40.236 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:40.237 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:40.238 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 31, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:40.239 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:40.240 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-04-25T13:36:40.241 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 223, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1558230, 2024-04-25T13:36:40.242 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 223, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 223, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1558230, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-04-25T13:36:40.243 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-04-25T13:36:40.244 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:40.244 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-25T13:36:40.244 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-25T13:36:41.131 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T13:36:41.131 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:36:41.131 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T13:36:41.131 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T13:36:41.132 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T13:36:41.139 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:40 smithi044 ceph-mon[32590]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:41.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:40 smithi037 ceph-mon[29204]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:41.380 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:36:41.384 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:40 smithi069 ceph-mon[32534]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:43.058 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:42 smithi069 ceph-mon[32534]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:43.058 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001399176, 2024-04-25T13:36:43.059 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000034748, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000560842, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000716290, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000238763 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.060 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:43.061 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:43.061 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:43.061 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:43.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.061 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 219, 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 208, 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1588896, 2024-04-25T13:36:43.062 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 127424, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.043895913, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.013400931, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.033789700, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005489885, 2024-04-25T13:36:43.063 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 207, 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.011398733, 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000055066 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:43.064 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1588896, 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 127424 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:43.065 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 159, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 183968, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:43.066 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.011729747, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.007849413, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001660890, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 156, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015526426, 2024-04-25T13:36:43.067 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000099528 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:43.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 183968 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T13:36:43.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:43.070 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:43.071 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:43.072 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:43.073 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:43.074 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T13:36:43.075 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:43.076 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T13:36:43.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 221, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1558034, 2024-04-25T13:36:43.078 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 221, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 221, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1558034, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:43.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:43.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:43.080 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:43.080 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:43.080 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T13:36:43.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:42 smithi037 ceph-mon[29204]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:43.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:42 smithi044 ceph-mon[32590]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:43.733 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T13:36:43.733 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T13:36:43.733 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T13:36:43.733 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T13:36:43.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T13:36:43.970 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:36:45.074 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:44 smithi069 ceph-mon[32534]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:45.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:44 smithi037 ceph-mon[29204]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:45.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:44 smithi044 ceph-mon[32590]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:45.670 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001125561, 2024-04-25T13:36:45.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000313951, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:45.672 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 225, 2024-04-25T13:36:45.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 212, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1592384, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 129152, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.045862746, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.013461290, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.034965414, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.006102583, 2024-04-25T13:36:45.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 212, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.013752191, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000064868 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.675 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1592384, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 129152 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 159, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 183968, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:45.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.011664602, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.007530359, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001753733, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 156, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015157290, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000097162 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 183968 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T13:36:45.678 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:45.679 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:45.680 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:45.681 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:45.682 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 149017, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:45.683 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:45.684 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:45.685 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 223, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1558294, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 223, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 223, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1558294, 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:45.686 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:45.687 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:45.687 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:45.687 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:45.687 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:45.687 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T13:36:46.348 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T13:36:46.348 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T13:36:46.348 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T13:36:46.349 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T13:36:46.349 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T13:36:46.582 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:36:47.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:46 smithi037 ceph-mon[29204]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:47.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:46 smithi069 ceph-mon[32534]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:47.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:46 smithi044 ceph-mon[32590]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:48.352 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T13:36:48.352 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:48.352 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 1555, 2024-04-25T13:36:48.352 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 1554, 2024-04-25T13:36:48.352 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 554247, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 493358, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 2, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.158392632, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.051823398, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.039993101, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.051267541, 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:48.353 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 1553, 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.065337165, 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000042071 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 241, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 229, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1610408, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 131690, 2024-04-25T13:36:48.355 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.046737824, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.014729843, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.035173129, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005502780, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 228, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015000616, 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000065792 2024-04-25T13:36:48.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1608752, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 130352 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.357 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 738, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 895, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 181447, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 678977, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 4, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.082510324, 2024-04-25T13:36:48.358 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.032705932, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.016178593, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.024554056, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 892, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.049342753, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000055316 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:48.359 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 444928 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 1, 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T13:36:48.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-MDSRank": { 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3741, 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.112621942, 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000030104 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.361 INFO:teuthology.orchestra.run.smithi069.stdout: "finisher-PurgeQueue": { 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "queue_len": 0, 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "complete_latency": { 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.362 INFO:teuthology.orchestra.run.smithi069.stdout: "mds": { 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "request": 0, 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "reply": 0, 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "reply_latency": { 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.363 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_reply": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "forward": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_complete": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_fetch_keys": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_commit": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_split": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_merge": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes": 12, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_top": 0, 2024-04-25T13:36:48.364 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_bottom": 12, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pin_tail": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_pinned": 12, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_expired": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "inodes_with_caps": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "caps": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "subtrees": 2, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_hit": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_forward": 0, 2024-04-25T13:36:48.365 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_discover": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_dir_fetch": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_remote_ino": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "traverse_lock": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "load_cent": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "q": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "exported": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "exported_inodes": 0, 2024-04-25T13:36:48.366 INFO:teuthology.orchestra.run.smithi069.stdout: "imported": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "imported_inodes": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_dir_fetch": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_backtrace_fetch": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "openino_peer_discover": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rfiles": 4, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rbytes": 234, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "root_rsnaps": 6, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T13:36:48.367 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_set_tag": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_inotable_repaired": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_inodes": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_file_inodes": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_inode_file_caps": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T13:36:48.368 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_grant": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_caps_dirty": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_cap_release": 0, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "process_request_cap_release": 0 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_cache": { 2024-04-25T13:36:48.369 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update": 0, 2024-04-25T13:36:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_update_receipt": 0, 2024-04-25T13:36:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_try_discover": 0, 2024-04-25T13:36:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_send_discover": 0, 2024-04-25T13:36:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "dir_handle_discover": 0, 2024-04-25T13:36:48.370 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_delayed": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "num_strays_enqueuing": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_created": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_enqueued": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_reintegrated": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "strays_migrated": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_processing": 0, 2024-04-25T13:36:48.371 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_enqueued": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "num_recovering_prioritized": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_started": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "recovery_completed": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_path": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_quiesce_inode": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_exportdir": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_flush": 0, 2024-04-25T13:36:48.372 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragmentdir": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_fragstats": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "ireq_inodestats": 0 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_log": { 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "evlrg": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "evadd": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "evex": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "evtrm": 0, 2024-04-25T13:36:48.373 INFO:teuthology.orchestra.run.smithi069.stdout: "ev": 101, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "evexg": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "evexd": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segadd": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segex": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segtrm": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "seg": 2, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segmjr": 2, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segexg": 0, 2024-04-25T13:36:48.374 INFO:teuthology.orchestra.run.smithi069.stdout: "segexd": 0, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "expos": 4194304, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "wrpos": 4283532, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "rdpos": 4283420, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "jlat": { 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.375 INFO:teuthology.orchestra.run.smithi069.stdout: "replayed": 101 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_mem": { 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "ino": 13, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "ino+": 13, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "ino-": 0, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "dir": 8, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "dir+": 8, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "dir-": 0, 2024-04-25T13:36:48.376 INFO:teuthology.orchestra.run.smithi069.stdout: "dn": 12, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "dn+": 12, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "dn-": 0, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "cap": 0, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "cap+": 0, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "cap-": 0, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "rss": 50528, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "heap": 215416 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.377 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_server": { 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_client_request": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "dispatch_server_request": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_request": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_client_session": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "handle_peer_request": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "req_create_latency": { 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.378 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getattr_latency": { 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getfilelock_latency": { 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.379 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "req_link_latency": { 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookup_latency": { 2024-04-25T13:36:48.380 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookuphash_latency": { 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.381 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupino_latency": { 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupname_latency": { 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupparent_latency": { 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lookupsnap_latency": { 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.383 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "req_lssnap_latency": { 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mkdir_latency": { 2024-04-25T13:36:48.384 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mknod_latency": { 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.385 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: "req_mksnap_latency": { 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "req_open_latency": { 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "req_readdir_latency": { 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.387 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rename_latency": { 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "req_renamesnap_latency": { 2024-04-25T13:36:48.388 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "req_snapdiff_latency": { 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmdir_latency": { 2024-04-25T13:36:48.389 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmsnap_latency": { 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "req_rmxattr_latency": { 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setattr_latency": { 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setdirlayout_latency": { 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setfilelock_latency": { 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.392 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setlayout_latency": { 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "req_setxattr_latency": { 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.393 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "req_symlink_latency": { 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "req_unlink_latency": { 2024-04-25T13:36:48.394 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_revoke_eviction": 0, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "cap_acquisition_throttle": 0, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "req_getvxattr_latency": { 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.395 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_sessions": { 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "session_count": 2, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "session_add": 5, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "session_remove": 3, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_open": 4, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "sessions_stale": 0, 2024-04-25T13:36:48.396 INFO:teuthology.orchestra.run.smithi069.stdout: "total_load": 0, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "average_load": 0, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "avg_session_uptime": 2866, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "mdthresh_evicted": 0 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:48.397 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:48.398 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:48.399 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:48.400 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 190431, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 52, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:48.401 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 8224, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 171, 2024-04-25T13:36:48.402 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 47746, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 220, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:48.403 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "objecter": { 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_active": 0, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_laggy": 0, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send": 2290, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_send_bytes": 26, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_resend": 4, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_reply": 2290, 2024-04-25T13:36:48.404 INFO:teuthology.orchestra.run.smithi069.stdout: "op_latency": { 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2290, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 1.599966017, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000698675 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "op_inflight": 0, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "oplen_avg": { 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 2290, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 2291 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.405 INFO:teuthology.orchestra.run.smithi069.stdout: "op": 2290, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "op_r": 2290, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "op_w": 0, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "op_rmw": 0, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "op_pg": 0, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_stat": 1487, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_create": 0, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_read": 800, 2024-04-25T13:36:48.406 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_write": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writefull": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_writesame": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_append": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_zero": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_truncate": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_delete": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_mapext": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_sparse_read": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_clonerange": 0, 2024-04-25T13:36:48.407 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_getxattr": 2, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_setxattr": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_cmpxattr": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_rmxattr": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_resetxattrs": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_call": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_watch": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_notify": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T13:36:48.408 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_pgls_filter": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "osdop_other": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_active": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_send": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_resend": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "linger_ping": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_active": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_send": 0, 2024-04-25T13:36:48.409 INFO:teuthology.orchestra.run.smithi069.stdout: "poolop_resend": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_active": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_send": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "poolstat_resend": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_active": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_send": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "statfs_resend": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "command_active": 0, 2024-04-25T13:36:48.410 INFO:teuthology.orchestra.run.smithi069.stdout: "command_send": 0, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "command_resend": 0, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "map_epoch": 100, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "map_full": 0, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "map_inc": 11, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_sessions": 5, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_open": 5, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_session_close": 0, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_laggy": 0, 2024-04-25T13:36:48.411 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_wr": 0, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_rd": 2, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_del": 0 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "oft": { 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_objs": 0, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_kv_pairs": 0, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_updates": 0, 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: "omap_total_removes": 0 2024-04-25T13:36:48.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "osd-slow-ops": { 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "slow_ops_count": 0 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "purge_queue": { 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops": 0, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing": 0, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executing_high_water": 0, 2024-04-25T13:36:48.413 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed_ops": 0, 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_executed": 0, 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "pq_item_in_journal": 0 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T13:36:48.414 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 2530, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 2103343, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 2530, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2530, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2103343, 2024-04-25T13:36:48.415 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_bytes": { 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:48.416 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 2290, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 89238, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2290, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 89238, 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:48.417 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-objecter_ops": { 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 1024, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:48.418 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 2290, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 2290, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 2290, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 2290, 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:48.419 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle": { 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:48.420 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-04-25T13:36:48.421 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-write_buf_throttle-0x5594e0b9c6e0": { 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 3758096384, 2024-04-25T13:36:48.422 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 0, 2024-04-25T13:36:48.423 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 0, 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:48.424 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T13:36:49.128 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:48 smithi069 ceph-mon[32534]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:49.156 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T13:36:49.156 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T13:36:49.156 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T13:36:49.156 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T13:36:49.156 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid d766aeb0-0305-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T13:36:49.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:48 smithi037 ceph-mon[29204]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:49.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:48 smithi044 ceph-mon[32590]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:49.402 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T13:36:50.892 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 2944, 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-04-25T13:36:50.893 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001391934, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000052708, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000513861, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-04-25T13:36:50.894 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000669402, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000223134 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T13:36:50.895 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 225, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 214, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1590528, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 131264, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-04-25T13:36:50.896 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.047555105, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.014496047, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.035354229, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.005781235, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 213, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.012453351, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000058466 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.897 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1590528, 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 131264 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T13:36:50.898 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 2, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 163, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 320, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 184672, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.011326151, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.007731849, 2024-04-25T13:36:50.899 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.001356493, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 160, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.018682699, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000116766 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:50.900 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 184672 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-04-25T13:36:50.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-04-25T13:36:50.903 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-04-25T13:36:50.904 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T13:36:50.905 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148977, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-04-25T13:36:50.906 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-04-25T13:36:50.907 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T13:36:50.908 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 227, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1558814, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 227, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-04-25T13:36:50.909 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 227, 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1558814, 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-04-25T13:36:50.910 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-04-25T13:36:51.135 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:50 smithi069 ceph-mon[32534]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:51.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:50 smithi037 ceph-mon[29204]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:51.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:50 smithi044 ceph-mon[32590]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:51.543 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T13:36:51.543 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:36:51.543 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T13:36:51.544 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T13:36:51.554 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T13:36:51.554 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:36:51.554 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:36:51.575 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T13:36:51.575 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:36:51.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:36:51.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 -- 172.21.15.37:0/3916840542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069250 msgr2=0x7f5b78069630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:51.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/3916840542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069250 0x7f5b78069630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009ca0 tx=0x7f5b6802a950 comp rx=0 tx=0).stop 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 -- 172.21.15.37:0/3916840542 shutdown_connections 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/3916840542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b7810ec50 0x7f5b78111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/3916840542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b78069b70 0x7f5b7810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/3916840542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069250 0x7f5b78069630 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 -- 172.21.15.37:0/3916840542 >> 172.21.15.37:0/3916840542 conn(0x7f5b78077c40 msgr2=0x7f5b78078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 -- 172.21.15.37:0/3916840542 shutdown_connections 2024-04-25T13:36:51.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.791+0000 7f5b7cbad640 1 -- 172.21.15.37:0/3916840542 wait complete. 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 Processor -- start 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 -- start start 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b78069250 0x7f5b781a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 0x7f5b7819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781143f0 con 0x7f5b78069b70 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78114270 con 0x7f5b78069250 2024-04-25T13:36:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b7cbad640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78114570 con 0x7f5b7810ec50 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b75d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b75d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43814/0 (socket says 172.21.15.37:43814) 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.792+0000 7f5b75d9b640 1 -- 172.21.15.37:0/1897412113 learned_addr learned my addr 172.21.15.37:0/1897412113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b7659c640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b78069250 0x7f5b781a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b76d9d640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 0x7f5b7819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 -- 172.21.15.37:0/1897412113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 msgr2=0x7f5b7819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 0x7f5b7819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 -- 172.21.15.37:0/1897412113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b78069250 msgr2=0x7f5b781a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b78069250 0x7f5b781a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 -- 172.21.15.37:0/1897412113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7819afa0 con 0x7f5b78069b70 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b76d9d640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 0x7f5b7819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b75d9b640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00e9f0 tx=0x7f5b6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00cd80 con 0x7f5b78069b70 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7819b290 con 0x7f5b78069b70 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c0054d0 con 0x7f5b78069b70 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c010690 con 0x7f5b78069b70 2024-04-25T13:36:51.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.793+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781a74b0 con 0x7f5b78069b70 2024-04-25T13:36:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.794+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b78101470 con 0x7f5b78069b70 2024-04-25T13:36:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.796+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b6c004e30 con 0x7f5b78069b70 2024-04-25T13:36:51.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.796+0000 7f5b677fe640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 0x7f5b600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.796+0000 7f5b7659c640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 0x7f5b600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.796+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f5b6c0d0f00 con 0x7f5b78069b70 2024-04-25T13:36:51.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.797+0000 7f5b7659c640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 0x7f5b600a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b68004ad0 tx=0x7f5b68004a60 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:51.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.800+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b6c053ff0 con 0x7f5b78069b70 2024-04-25T13:36:51.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.990+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b78109de0 con 0x7f5b78069b70 2024-04-25T13:36:51.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.991+0000 7f5b677fe640 1 -- 172.21.15.37:0/1897412113 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23072 (secure 0 0 0) 0x7f5b6c09d570 con 0x7f5b78069b70 2024-04-25T13:36:51.992 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:51.992 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:36:10.291364+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1479124970":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/4004466629":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/774754895":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/683134189":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.993+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 msgr2=0x7f5b600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.993+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 0x7f5b600a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b68004ad0 tx=0x7f5b68004a60 comp rx=0 tx=0).stop 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.993+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 msgr2=0x7f5b781a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.993+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00e9f0 tx=0x7f5b6c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 shutdown_connections 2024-04-25T13:36:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5b600a3d20 0x7f5b600a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b7810ec50 0x7f5b7819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b78069b70 0x7f5b781a0ba0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 --2- 172.21.15.37:0/1897412113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b78069250 0x7f5b781a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 >> 172.21.15.37:0/1897412113 conn(0x7f5b78077c40 msgr2=0x7f5b78110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 shutdown_connections 2024-04-25T13:36:51.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:51.994+0000 7f5b7cbad640 1 -- 172.21.15.37:0/1897412113 wait complete. 2024-04-25T13:36:52.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:19:35.726861+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:24:08.960973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:24:10.167382+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:36:52.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 -- 172.21.15.37:0/2610543608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 msgr2=0x7fed001035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 --2- 172.21.15.37:0/2610543608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed001035a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009900 tx=0x7fecf002a480 comp rx=0 tx=0).stop 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 -- 172.21.15.37:0/2610543608 shutdown_connections 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 --2- 172.21.15.37:0/2610543608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed0010c500 0x7fed0010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 --2- 172.21.15.37:0/2610543608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00103ae0 0x7fed0010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 --2- 172.21.15.37:0/2610543608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed001035a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.111+0000 7fed050fa640 1 -- 172.21.15.37:0/2610543608 >> 172.21.15.37:0/2610543608 conn(0x7fed00078000 msgr2=0x7fed00078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.112+0000 7fed050fa640 1 -- 172.21.15.37:0/2610543608 shutdown_connections 2024-04-25T13:36:52.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.112+0000 7fed050fa640 1 -- 172.21.15.37:0/2610543608 wait complete. 2024-04-25T13:36:52.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.112+0000 7fed050fa640 1 Processor -- start 2024-04-25T13:36:52.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.112+0000 7fed050fa640 1 -- start start 2024-04-25T13:36:52.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00103ae0 0x7fed00101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0010c500 0x7fed00101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed00112170 con 0x7fed001031c0 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed00111ff0 con 0x7fed0010c500 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fed050fa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fed001122f0 con 0x7fed00103ae0 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43828/0 (socket says 172.21.15.37:43828) 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 -- 172.21.15.37:0/1059552527 learned_addr learned my addr 172.21.15.37:0/1059552527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecff59e640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0010c500 0x7fed00101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfe59c640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00103ae0 0x7fed00101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 -- 172.21.15.37:0/1059552527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00103ae0 msgr2=0x7fed00101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00103ae0 0x7fed00101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 -- 172.21.15.37:0/1059552527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0010c500 msgr2=0x7fed00101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0010c500 0x7fed00101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 -- 172.21.15.37:0/1059552527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed001a6b90 con 0x7fed001031c0 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fecfed9d640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fecf0005940 tx=0x7fecf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.113+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf003f030 con 0x7fed001031c0 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.114+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed001a6e20 con 0x7fed001031c0 2024-04-25T13:36:52.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.114+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf0003cb0 con 0x7fed001031c0 2024-04-25T13:36:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.114+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0035070 con 0x7fed001031c0 2024-04-25T13:36:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.114+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed001a7290 con 0x7fed001031c0 2024-04-25T13:36:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.115+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed00068e00 con 0x7fed001031c0 2024-04-25T13:36:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.116+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecf0042030 con 0x7fed001031c0 2024-04-25T13:36:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.116+0000 7fece7fff640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 0x7fece80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.116+0000 7fecfe59c640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 0x7fece80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.116+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fecf00f1070 con 0x7fed001031c0 2024-04-25T13:36:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.117+0000 7fecfe59c640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 0x7fece80a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005fd0 tx=0x7fecf4005850 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.121+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fecf00bd6e0 con 0x7fed001031c0 2024-04-25T13:36:52.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.307+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fed00107680 con 0x7fed001031c0 2024-04-25T13:36:52.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.308+0000 7fece7fff640 1 -- 172.21.15.37:0/1059552527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fecf00bcfe0 con 0x7fed001031c0 2024-04-25T13:36:52.308 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T13:36:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 msgr2=0x7fece80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 0x7fece80a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecf4005fd0 tx=0x7fecf4005850 comp rx=0 tx=0).stop 2024-04-25T13:36:52.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 msgr2=0x7fed00100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fecf0005940 tx=0x7fecf00042f0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 shutdown_connections 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fece80a3d20 0x7fece80a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed0010c500 0x7fed00101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed00103ae0 0x7fed00101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 --2- 172.21.15.37:0/1059552527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed001031c0 0x7fed00100e10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 >> 172.21.15.37:0/1059552527 conn(0x7fed00078000 msgr2=0x7fed000fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 shutdown_connections 2024-04-25T13:36:52.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.310+0000 7fed050fa640 1 -- 172.21.15.37:0/1059552527 wait complete. 2024-04-25T13:36:52.321 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/4075079343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18101050 msgr2=0x7f5d18101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/4075079343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18101050 0x7f5d18101430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d1002a410 comp rx=0 tx=0).stop 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/4075079343 shutdown_connections 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/4075079343 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18106120 0x7f5d1810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/4075079343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d18101a00 0x7f5d18105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/4075079343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18101050 0x7f5d18101430 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.425+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/4075079343 >> 172.21.15.37:0/4075079343 conn(0x7f5d18077fa0 msgr2=0x7f5d180783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/4075079343 shutdown_connections 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/4075079343 wait complete. 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 Processor -- start 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- start start 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 0x7f5d18197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d1819bf20 0x7f5d1819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18111080 con 0x7f5d18106120 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18110f00 con 0x7f5d18101050 2024-04-25T13:36:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1f8f5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18111200 con 0x7f5d1819bf20 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.426+0000 7f5d1ce90640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43852/0 (socket says 172.21.15.37:43852) 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 -- 172.21.15.37:0/3444047278 learned_addr learned my addr 172.21.15.37:0/3444047278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1de92640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d1819bf20 0x7f5d1819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1d691640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 0x7f5d18197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 -- 172.21.15.37:0/3444047278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d1819bf20 msgr2=0x7f5d1819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d1819bf20 0x7f5d1819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 -- 172.21.15.37:0/3444047278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 msgr2=0x7f5d18197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 0x7f5d18197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 -- 172.21.15.37:0/3444047278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1819ca10 con 0x7f5d18106120 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1d691640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 0x7f5d18197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1ce90640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d740 tx=0x7f5d0800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08017070 con 0x7f5d18106120 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0800de50 con 0x7f5d18106120 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.427+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1819cd00 con 0x7f5d18106120 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.428+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08004ce0 con 0x7f5d18106120 2024-04-25T13:36:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.428+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181a44f0 con 0x7f5d18106120 2024-04-25T13:36:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.429+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d18102520 con 0x7f5d18106120 2024-04-25T13:36:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.430+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d08004e80 con 0x7f5d18106120 2024-04-25T13:36:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.430+0000 7f5d0e7fc640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 0x7f5d040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.431+0000 7f5d1d691640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 0x7f5d040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.431+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f5d080d01b0 con 0x7f5d18106120 2024-04-25T13:36:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.431+0000 7f5d1d691640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 0x7f5d040a6260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d10000c00 tx=0x7f5d10031400 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.435+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d0809c7c0 con 0x7f5d18106120 2024-04-25T13:36:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.620+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5d18101ad0 con 0x7f5d18106120 2024-04-25T13:36:52.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.620+0000 7f5d0e7fc640 1 -- 172.21.15.37:0/3444047278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f5d0809c0c0 con 0x7f5d18106120 2024-04-25T13:36:52.620 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T13:36:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 msgr2=0x7f5d040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 0x7f5d040a6260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d10000c00 tx=0x7f5d10031400 comp rx=0 tx=0).stop 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 msgr2=0x7f5d18197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800d740 tx=0x7f5d0800dc00 comp rx=0 tx=0).stop 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 shutdown_connections 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f5d040a3da0 0x7f5d040a6260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5d1819bf20 0x7f5d1819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d18106120 0x7f5d18197ba0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 --2- 172.21.15.37:0/3444047278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d18101050 0x7f5d18197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 >> 172.21.15.37:0/3444047278 conn(0x7f5d18077fa0 msgr2=0x7f5d180ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 shutdown_connections 2024-04-25T13:36:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.622+0000 7f5d1f8f5640 1 -- 172.21.15.37:0/3444047278 wait complete. 2024-04-25T13:36:52.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:36:52.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 -- 172.21.15.37:0/256934353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 msgr2=0x7feebc112d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 --2- 172.21.15.37:0/256934353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc112d00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009930 tx=0x7feeb802a420 comp rx=0 tx=0).stop 2024-04-25T13:36:52.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 -- 172.21.15.37:0/256934353 shutdown_connections 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 --2- 172.21.15.37:0/256934353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1197e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 --2- 172.21.15.37:0/256934353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 0x7feebc10cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 --2- 172.21.15.37:0/256934353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc112d00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 -- 172.21.15.37:0/256934353 >> 172.21.15.37:0/256934353 conn(0x7feebc06d100 msgr2=0x7feebc10af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 -- 172.21.15.37:0/256934353 shutdown_connections 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.738+0000 7feec3c62640 1 -- 172.21.15.37:0/256934353 wait complete. 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 Processor -- start 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 -- start start 2024-04-25T13:36:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 0x7feebc1a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feebc11cbe0 con 0x7feebc112920 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feebc11ca60 con 0x7feebc10d2d0 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec3c62640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feebc11cd60 con 0x7feebc10c910 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43862/0 (socket says 172.21.15.37:43862) 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 -- 172.21.15.37:0/2202210509 learned_addr learned my addr 172.21.15.37:0/2202210509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec19fe640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 0x7feebc1a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec11fd640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 -- 172.21.15.37:0/2202210509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 msgr2=0x7feebc1a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 0x7feebc1a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 -- 172.21.15.37:0/2202210509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 msgr2=0x7feebc1a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec21ff640 1 -- 172.21.15.37:0/2202210509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc1a8150 con 0x7feebc112920 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.739+0000 7feec11fd640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feec21ff640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e960 tx=0x7feeb400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4004d20 con 0x7feebc112920 2024-04-25T13:36:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb4005020 con 0x7feebc112920 2024-04-25T13:36:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc1a8440 con 0x7feebc112920 2024-04-25T13:36:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40116d0 con 0x7feebc112920 2024-04-25T13:36:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.740+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc1afcd0 con 0x7feebc112920 2024-04-25T13:36:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.741+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc068e00 con 0x7feebc112920 2024-04-25T13:36:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.742+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feeb40051d0 con 0x7feebc112920 2024-04-25T13:36:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.743+0000 7feeb2ffd640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 0x7feea80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.743+0000 7feec19fe640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 0x7feea80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.743+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7feeb40d0ca0 con 0x7feebc112920 2024-04-25T13:36:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.743+0000 7feec19fe640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 0x7feea80a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009900 tx=0x7feeb8000bc0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.746+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feeb4016030 con 0x7feebc112920 2024-04-25T13:36:52.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.933+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feebc02d5c0 con 0x7feebc112920 2024-04-25T13:36:52.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.934+0000 7feeb2ffd640 1 -- 172.21.15.37:0/2202210509 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7feeb409d330 con 0x7feebc112920 2024-04-25T13:36:52.934 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T13:36:52.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 msgr2=0x7feea80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 0x7feea80a6260 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009900 tx=0x7feeb8000bc0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 msgr2=0x7feebc1a7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feeb400e960 tx=0x7feeb400ee20 comp rx=0 tx=0).stop 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 shutdown_connections 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7feea80a3da0 0x7feea80a6260 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feebc112920 0x7feebc1a7a20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feebc10d2d0 0x7feebc1a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 --2- 172.21.15.37:0/2202210509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10c910 0x7feebc1a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 >> 172.21.15.37:0/2202210509 conn(0x7feebc06d100 msgr2=0x7feebc10f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 shutdown_connections 2024-04-25T13:36:52.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:52.936+0000 7feec3c62640 1 -- 172.21.15.37:0/2202210509 wait complete. 2024-04-25T13:36:52.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 -- 172.21.15.37:0/3497514460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103250 msgr2=0x7f4c0c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/3497514460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103250 0x7f4c0c103630 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009900 tx=0x7f4bfc02a480 comp rx=0 tx=0).stop 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 -- 172.21.15.37:0/3497514460 shutdown_connections 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/3497514460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c10c620 0x7f4c0c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/3497514460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c103b70 0x7f4c0c10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/3497514460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103250 0x7f4c0c103630 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 -- 172.21.15.37:0/3497514460 >> 172.21.15.37:0/3497514460 conn(0x7f4c0c078000 msgr2=0x7f4c0c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 -- 172.21.15.37:0/3497514460 shutdown_connections 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.050+0000 7f4c11ae8640 1 -- 172.21.15.37:0/3497514460 wait complete. 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 Processor -- start 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 -- start start 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c103250 0x7f4c0c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 0x7f4c0c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c112270 con 0x7f4c0c103b70 2024-04-25T13:36:53.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1120f0 con 0x7f4c0c103250 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c11ae8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1123f0 con 0x7f4c0c10c620 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43882/0 (socket says 172.21.15.37:43882) 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.051+0000 7f4c0b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c103250 0x7f4c0c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 -- 172.21.15.37:0/827307512 learned_addr learned my addr 172.21.15.37:0/827307512 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0bfff640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 0x7f4c0c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 -- 172.21.15.37:0/827307512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 msgr2=0x7f4c0c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 0x7f4c0c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 -- 172.21.15.37:0/827307512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c103250 msgr2=0x7f4c0c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c103250 0x7f4c0c19e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 -- 172.21.15.37:0/827307512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c198f70 con 0x7f4c0c103b70 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0bfff640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 0x7f4c0c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c0affd640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c970 tx=0x7f4c0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00007bb0 con 0x7f4c0c103b70 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c199260 con 0x7f4c0c103b70 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00007d50 con 0x7f4c0c103b70 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0000f450 con 0x7f4c0c103b70 2024-04-25T13:36:53.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.052+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1a5570 con 0x7f4c0c103b70 2024-04-25T13:36:53.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.053+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c06aeb0 con 0x7f4c0c103b70 2024-04-25T13:36:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.055+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c00004f00 con 0x7f4c0c103b70 2024-04-25T13:36:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.055+0000 7f4c08ff9640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 0x7f4bf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.055+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4c000d0920 con 0x7f4c0c103b70 2024-04-25T13:36:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.055+0000 7f4c0b7fe640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 0x7f4bf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.056+0000 7f4c0b7fe640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 0x7f4bf40a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc031440 tx=0x7f4bfc005b50 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:53.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.059+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c00014030 con 0x7f4c0c103b70 2024-04-25T13:36:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.257+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c0c199be0 con 0x7f4c0c103b70 2024-04-25T13:36:53.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.257+0000 7f4c08ff9640 1 -- 172.21.15.37:0/827307512 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+943 (secure 0 0 0) 0x7f4c0009cf90 con 0x7f4c0c103b70 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1479124970 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4004466629 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2819711067 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/774754895 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/3707854772 2024-04-26T13:17:05.642821+0000 2024-04-25T13:36:53.258 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/3707854772 2024-04-26T13:17:05.642821+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3175650939 2024-04-26T13:17:05.642821+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1473000576 2024-04-26T13:16:37.834096+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2107975660 2024-04-26T13:16:37.834096+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/755315013 2024-04-26T13:17:05.642821+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1321688497 2024-04-26T13:17:05.642821+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4026748437 2024-04-26T13:16:37.834096+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2456399257 2024-04-26T13:16:37.834096+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/683134189 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2819711067 2024-04-26T13:26:11.228097+0000 2024-04-25T13:36:53.259 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2107975660 2024-04-26T13:16:37.834096+0000 2024-04-25T13:36:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-04-25T13:36:53.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 msgr2=0x7f4bf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 0x7f4bf40a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc031440 tx=0x7f4bfc005b50 comp rx=0 tx=0).stop 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 msgr2=0x7f4c0c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000c970 tx=0x7f4c0000ce30 comp rx=0 tx=0).stop 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 shutdown_connections 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f4bf40a3d20 0x7f4bf40a61e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c0c10c620 0x7f4c0c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c0c103b70 0x7f4c0c19eb30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 --2- 172.21.15.37:0/827307512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c0c103250 0x7f4c0c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 >> 172.21.15.37:0/827307512 conn(0x7f4c0c078000 msgr2=0x7f4c0c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:53.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 shutdown_connections 2024-04-25T13:36:53.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.259+0000 7f4c11ae8640 1 -- 172.21.15.37:0/827307512 wait complete. 2024-04-25T13:36:53.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:52 smithi037 ceph-mon[29204]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:53.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:52 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1897412113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:53.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:52 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1059552527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.270 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:52 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3444047278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.270 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:52 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2202210509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.271 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T13:36:53.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:36:53.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T13:36:53.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:36:53.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:36:53.301 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:36:53.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:36:53.361 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T13:36:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:52 smithi069 ceph-mon[32534]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:52 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1897412113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:52 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1059552527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:52 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3444047278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:52 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2202210509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:52 smithi044 ceph-mon[32590]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T13:36:53.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:52 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1897412113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:53.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:52 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1059552527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:52 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3444047278' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:52 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2202210509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T13:36:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:36:53.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:36:53.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:36:53.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:36:53.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T13:36:53.580 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T13:36:53.591 INFO:tasks.cephadm:Teardown begin 2024-04-25T13:36:53.591 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:36:53.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:36:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 -- 172.21.15.37:0/743562559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98101a50 msgr2=0x7f8a98105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/743562559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98101a50 0x7f8a98105a90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009900 tx=0x7f8a8c02a410 comp rx=0 tx=0).stop 2024-04-25T13:36:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 -- 172.21.15.37:0/743562559 shutdown_connections 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/743562559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 0x7f8a9810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/743562559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a98101a50 0x7f8a98105a90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/743562559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 0x7f8a98101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 -- 172.21.15.37:0/743562559 >> 172.21.15.37:0/743562559 conn(0x7f8a98077f50 msgr2=0x7f8a98078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 -- 172.21.15.37:0/743562559 shutdown_connections 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.695+0000 7f8a9dca0640 1 -- 172.21.15.37:0/743562559 wait complete. 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 Processor -- start 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 -- start start 2024-04-25T13:36:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 0x7f8a98197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 0x7f8a98197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981110b0 con 0x7f8a9819bef0 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98110f30 con 0x7f8a98106170 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a9dca0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98111230 con 0x7f8a981010a0 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43904/0 (socket says 172.21.15.37:43904) 2024-04-25T13:36:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a97fff640 1 -- 172.21.15.37:0/2453476920 learned_addr learned my addr 172.21.15.37:0/2453476920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a977fe640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 0x7f8a98197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.696+0000 7f8a96ffd640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 0x7f8a98197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 -- 172.21.15.37:0/2453476920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 msgr2=0x7f8a98197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 0x7f8a98197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 -- 172.21.15.37:0/2453476920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 msgr2=0x7f8a98197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 0x7f8a98197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 -- 172.21.15.37:0/2453476920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9819c9e0 con 0x7f8a9819bef0 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a97fff640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000eaa0 tx=0x7f8a9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000cdb0 con 0x7f8a9819bef0 2024-04-25T13:36:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9819ccd0 con 0x7f8a9819bef0 2024-04-25T13:36:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90005020 con 0x7f8a9819bef0 2024-04-25T13:36:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900116d0 con 0x7f8a9819bef0 2024-04-25T13:36:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.697+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a981a4490 con 0x7f8a9819bef0 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.698+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a98069250 con 0x7f8a9819bef0 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.699+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a900051d0 con 0x7f8a9819bef0 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.700+0000 7f8a94ff9640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 0x7f8a800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.700+0000 7f8a977fe640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 0x7f8a800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.700+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f8a900d0df0 con 0x7f8a9819bef0 2024-04-25T13:36:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.700+0000 7f8a977fe640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 0x7f8a800a61e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009bd0 tx=0x7f8a88005f90 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:53.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.703+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a90016030 con 0x7f8a9819bef0 2024-04-25T13:36:53.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.930+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a98109400 con 0x7f8a9819bef0 2024-04-25T13:36:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.931+0000 7f8a94ff9640 1 -- 172.21.15.37:0/2453476920 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7f8a9009d400 con 0x7f8a9819bef0 2024-04-25T13:36:53.931 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:53.931 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:36:53.931 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:36:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 msgr2=0x7f8a800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 0x7f8a800a61e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009bd0 tx=0x7f8a88005f90 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 msgr2=0x7f8a9819c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000eaa0 tx=0x7f8a9000ef60 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 shutdown_connections 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f8a800a3d20 0x7f8a800a61e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a9819bef0 0x7f8a9819c3a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98106170 0x7f8a98197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 --2- 172.21.15.37:0/2453476920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8a981010a0 0x7f8a98197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:53.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 >> 172.21.15.37:0/2453476920 conn(0x7f8a98077f50 msgr2=0x7f8a980ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:53.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 shutdown_connections 2024-04-25T13:36:53.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:53.933+0000 7f8a9dca0640 1 -- 172.21.15.37:0/2453476920 wait complete. 2024-04-25T13:36:53.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:36:54.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1136902459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc101a50 msgr2=0x7fe5bc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1136902459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc101a50 0x7fe5bc105a90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009900 tx=0x7fe5a402a410 comp rx=0 tx=0).stop 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1136902459 shutdown_connections 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1136902459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc106170 0x7fe5bc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1136902459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc101a50 0x7fe5bc105a90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1136902459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 0x7fe5bc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1136902459 >> 172.21.15.37:0/1136902459 conn(0x7fe5bc077f50 msgr2=0x7fe5bc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1136902459 shutdown_connections 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.047+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1136902459 wait complete. 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 Processor -- start 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 -- start start 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 0x7fe5bc10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 0x7fe5bc1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1121f0 con 0x7fe5bc106170 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc112070 con 0x7fe5bc1010a0 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5c10e1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc112370 con 0x7fe5bc10f150 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43924/0 (socket says 172.21.15.37:43924) 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.048+0000 7fe5b3fff640 1 -- 172.21.15.37:0/1224505041 learned_addr learned my addr 172.21.15.37:0/1224505041 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5bb59e640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 0x7fe5bc1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5bad9d640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 0x7fe5bc10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 -- 172.21.15.37:0/1224505041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 msgr2=0x7fe5bc1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 0x7fe5bc1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 -- 172.21.15.37:0/1224505041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 msgr2=0x7fe5bc10e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 0x7fe5bc10e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 -- 172.21.15.37:0/1224505041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc1a6fc0 con 0x7fe5bc106170 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5bb59e640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 0x7fe5bc1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b3fff640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4006010 tx=0x7fe5a4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a403f030 con 0x7fe5bc106170 2024-04-25T13:36:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1a7250 con 0x7fe5bc106170 2024-04-25T13:36:54.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a4003aa0 con 0x7fe5bc106170 2024-04-25T13:36:54.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4035070 con 0x7fe5bc106170 2024-04-25T13:36:54.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.049+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a7740 con 0x7fe5bc106170 2024-04-25T13:36:54.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.050+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc069250 con 0x7fe5bc106170 2024-04-25T13:36:54.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.051+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe5a4042030 con 0x7fe5bc106170 2024-04-25T13:36:54.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.052+0000 7fe5b8d99640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 0x7fe5a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.052+0000 7fe5bad9d640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 0x7fe5a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.052+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fe5a40f0d90 con 0x7fe5bc106170 2024-04-25T13:36:54.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.052+0000 7fe5bad9d640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 0x7fe5a80a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009b60 tx=0x7fe5ac005f90 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.056+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5a40bd3a0 con 0x7fe5bc106170 2024-04-25T13:36:54.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.280+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5bc109400 con 0x7fe5bc106170 2024-04-25T13:36:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.281+0000 7fe5b8d99640 1 -- 172.21.15.37:0/1224505041 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11792 (secure 0 0 0) 0x7fe5a40bcca0 con 0x7fe5bc106170 2024-04-25T13:36:54.282 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:54.282 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1814974020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1814974020},{"type":"v1","addr":"172.21.15.37:6835","nonce":1814974020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/3506634050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":3506634050},{"type":"v1","addr":"172.21.15.37:6837","nonce":3506634050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/3019792631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":3019792631},{"type":"v1","addr":"172.21.15.37:6839","nonce":3019792631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/1127100718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":1127100718},{"type":"v1","addr":"172.21.15.37:6841","nonce":1127100718}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/2542542028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2542542028},{"type":"v1","addr":"172.21.15.44:6833","nonce":2542542028}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6833/579371697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":579371697},{"type":"v1","addr":"172.21.15.69:6833","nonce":579371697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1266567676","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1266567676},{"type":"v1","addr":"172.21.15.44:6835","nonce":1266567676}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/690443752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":690443752},{"type":"v1","addr":"172.21.15.44:6837","nonce":690443752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/2452024894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":2452024894},{"type":"v1","addr":"172.21.15.69:6835","nonce":2452024894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/2736481323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":2736481323},{"type":"v1","addr":"172.21.15.69:6837","nonce":2736481323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T13:24:11.206089+0000","modified":"2024-04-25T13:24:15.243835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24520":{"gid":24520,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6839/1356630935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1356630935},{"type":"v1","addr":"172.21.15.69:6839","nonce":1356630935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24527":{"gid":24527,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.44:6839/1296390680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":1296390680},{"type":"v1","addr":"172.21.15.44:6839","nonce":1296390680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24527,"qdb_cluster":[24527]},"id":1}]} 2024-04-25T13:36:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 20 2024-04-25T13:36:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 msgr2=0x7fe5a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 0x7fe5a80a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009b60 tx=0x7fe5ac005f90 comp rx=0 tx=0).stop 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 msgr2=0x7fe5bc10ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4006010 tx=0x7fe5a4004060 comp rx=0 tx=0).stop 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 shutdown_connections 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fe5a80a3d20 0x7fe5a80a61e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5bc10f150 0x7fe5bc1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5bc106170 0x7fe5bc10ec10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 --2- 172.21.15.37:0/1224505041 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5bc1010a0 0x7fe5bc10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 >> 172.21.15.37:0/1224505041 conn(0x7fe5bc077f50 msgr2=0x7fe5bc0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 shutdown_connections 2024-04-25T13:36:54.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.283+0000 7fe5c10e1640 1 -- 172.21.15.37:0/1224505041 wait complete. 2024-04-25T13:36:54.290 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:53 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/827307512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:36:54.290 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:53 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2453476920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:54.293 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:53 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/827307512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:36:54.293 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:53 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2453476920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:54.294 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:36:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:53 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/827307512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T13:36:54.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:53 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2453476920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 -- 172.21.15.37:0/1123799629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79701010a0 msgr2=0x7f7970101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 --2- 172.21.15.37:0/1123799629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79701010a0 0x7f7970101480 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f796c009900 tx=0x7f796c02a410 comp rx=0 tx=0).stop 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 -- 172.21.15.37:0/1123799629 shutdown_connections 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 --2- 172.21.15.37:0/1123799629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970106170 0x7f797010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 --2- 172.21.15.37:0/1123799629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7970101a50 0x7f7970105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 --2- 172.21.15.37:0/1123799629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79701010a0 0x7f7970101480 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 -- 172.21.15.37:0/1123799629 >> 172.21.15.37:0/1123799629 conn(0x7f7970077f50 msgr2=0x7f7970078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 -- 172.21.15.37:0/1123799629 shutdown_connections 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.396+0000 7f797844e640 1 -- 172.21.15.37:0/1123799629 wait complete. 2024-04-25T13:36:54.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 Processor -- start 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 -- start start 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 0x7f797019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970198590 0x7f7970198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7970112250 con 0x7f7970106170 2024-04-25T13:36:54.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79701120d0 con 0x7f7970198590 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f797844e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79701123d0 con 0x7f79701010a0 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43936/0 (socket says 172.21.15.37:43936) 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 -- 172.21.15.37:0/3329774900 learned_addr learned my addr 172.21.15.37:0/3329774900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79761ea640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 0x7f797019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79769eb640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970198590 0x7f7970198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 -- 172.21.15.37:0/3329774900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 msgr2=0x7f797019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 0x7f797019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 -- 172.21.15.37:0/3329774900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970198590 msgr2=0x7f7970198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970198590 0x7f7970198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79759e9640 1 -- 172.21.15.37:0/3329774900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79701990b0 con 0x7f7970106170 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.397+0000 7f79761ea640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 0x7f797019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f79759e9640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f796000e9f0 tx=0x7f796000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796000ccc0 con 0x7f7970106170 2024-04-25T13:36:54.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79600054d0 con 0x7f7970106170 2024-04-25T13:36:54.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79701993a0 con 0x7f7970106170 2024-04-25T13:36:54.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960010430 con 0x7f7970106170 2024-04-25T13:36:54.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.398+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79701a55f0 con 0x7f7970106170 2024-04-25T13:36:54.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.399+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7970069250 con 0x7f7970106170 2024-04-25T13:36:54.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.400+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79600046c0 con 0x7f7970106170 2024-04-25T13:36:54.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.401+0000 7f79677fe640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 0x7f795c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.401+0000 7f79761ea640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 0x7f795c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.401+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f79600d0c80 con 0x7f7970106170 2024-04-25T13:36:54.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.401+0000 7f79761ea640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 0x7f795c0a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f796c031440 tx=0x7f796c005a20 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.405+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f796009d2f0 con 0x7f7970106170 2024-04-25T13:36:54.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.595+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7970199ac0 con 0x7f7970106170 2024-04-25T13:36:54.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.597+0000 7f79677fe640 1 -- 172.21.15.37:0/3329774900 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23072 (secure 0 0 0) 0x7f7970199ac0 con 0x7f7970106170 2024-04-25T13:36:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T13:36:54.598 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"d766aeb0-0305-11ef-bc93-c7b262605968","created":"2024-04-25T13:16:08.851582+0000","modified":"2024-04-25T13:36:10.291364+0000","last_up_change":"2024-04-25T13:23:20.457848+0000","last_in_change":"2024-04-25T13:22:59.656672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:19:35.726861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:24:08.960973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:24:10.167382+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f78d67e4-010a-4965-86af-acd038f6cdf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6803","nonce":3385916087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6805","nonce":3385916087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6809","nonce":3385916087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":3385916087},{"type":"v1","addr":"172.21.15.37:6807","nonce":3385916087}]},"public_addr":"172.21.15.37:6803/3385916087","cluster_addr":"172.21.15.37:6805/3385916087","heartbeat_back_addr":"172.21.15.37:6809/3385916087","heartbeat_front_addr":"172.21.15.37:6807/3385916087","state":["exists","up"]},{"osd":1,"uuid":"037053d7-433f-4d1a-9e7a-d44599f4a488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6801","nonce":1009231193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6803","nonce":1009231193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6807","nonce":1009231193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":1009231193},{"type":"v1","addr":"172.21.15.44:6805","nonce":1009231193}]},"public_addr":"172.21.15.44:6801/1009231193","cluster_addr":"172.21.15.44:6803/1009231193","heartbeat_back_addr":"172.21.15.44:6807/1009231193","heartbeat_front_addr":"172.21.15.44:6805/1009231193","state":["exists","up"]},{"osd":2,"uuid":"3317a25e-0c74-403e-94a3-fba65b8e6733","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6801","nonce":3261521434}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6803","nonce":3261521434}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6807","nonce":3261521434}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3261521434},{"type":"v1","addr":"172.21.15.69:6805","nonce":3261521434}]},"public_addr":"172.21.15.69:6801/3261521434","cluster_addr":"172.21.15.69:6803/3261521434","heartbeat_back_addr":"172.21.15.69:6807/3261521434","heartbeat_front_addr":"172.21.15.69:6805/3261521434","state":["exists","up"]},{"osd":3,"uuid":"97436b7d-2b40-4fbb-b523-4773cbf165f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6811","nonce":1255089911}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6813","nonce":1255089911}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6817","nonce":1255089911}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1255089911},{"type":"v1","addr":"172.21.15.37:6815","nonce":1255089911}]},"public_addr":"172.21.15.37:6811/1255089911","cluster_addr":"172.21.15.37:6813/1255089911","heartbeat_back_addr":"172.21.15.37:6817/1255089911","heartbeat_front_addr":"172.21.15.37:6815/1255089911","state":["exists","up"]},{"osd":4,"uuid":"748fab07-a621-4314-a001-a402dc9d7d08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6809","nonce":844548522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6811","nonce":844548522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6815","nonce":844548522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":844548522},{"type":"v1","addr":"172.21.15.44:6813","nonce":844548522}]},"public_addr":"172.21.15.44:6809/844548522","cluster_addr":"172.21.15.44:6811/844548522","heartbeat_back_addr":"172.21.15.44:6815/844548522","heartbeat_front_addr":"172.21.15.44:6813/844548522","state":["exists","up"]},{"osd":5,"uuid":"1dbcf0a2-1319-4cc8-9c58-d1d2a5520476","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6809","nonce":3958240806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6811","nonce":3958240806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6815","nonce":3958240806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":3958240806},{"type":"v1","addr":"172.21.15.69:6813","nonce":3958240806}]},"public_addr":"172.21.15.69:6809/3958240806","cluster_addr":"172.21.15.69:6811/3958240806","heartbeat_back_addr":"172.21.15.69:6815/3958240806","heartbeat_front_addr":"172.21.15.69:6813/3958240806","state":["exists","up"]},{"osd":6,"uuid":"d7fef8d2-092c-445a-aa3e-64878cfcd1a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6819","nonce":4082263223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6821","nonce":4082263223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6825","nonce":4082263223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":4082263223},{"type":"v1","addr":"172.21.15.37:6823","nonce":4082263223}]},"public_addr":"172.21.15.37:6819/4082263223","cluster_addr":"172.21.15.37:6821/4082263223","heartbeat_back_addr":"172.21.15.37:6825/4082263223","heartbeat_front_addr":"172.21.15.37:6823/4082263223","state":["exists","up"]},{"osd":7,"uuid":"d0f8c98d-4a36-4356-89c8-44fe3c5cfdcb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6817","nonce":3060345546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6819","nonce":3060345546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6823","nonce":3060345546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3060345546},{"type":"v1","addr":"172.21.15.44:6821","nonce":3060345546}]},"public_addr":"172.21.15.44:6817/3060345546","cluster_addr":"172.21.15.44:6819/3060345546","heartbeat_back_addr":"172.21.15.44:6823/3060345546","heartbeat_front_addr":"172.21.15.44:6821/3060345546","state":["exists","up"]},{"osd":8,"uuid":"6ddaeea5-1d9a-4d4b-b9ef-5d43723f8cb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6817","nonce":1610697891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6819","nonce":1610697891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6823","nonce":1610697891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1610697891},{"type":"v1","addr":"172.21.15.69:6821","nonce":1610697891}]},"public_addr":"172.21.15.69:6817/1610697891","cluster_addr":"172.21.15.69:6819/1610697891","heartbeat_back_addr":"172.21.15.69:6823/1610697891","heartbeat_front_addr":"172.21.15.69:6821/1610697891","state":["exists","up"]},{"osd":9,"uuid":"3c7a5005-401a-4bcb-b6fc-357793f24756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6827","nonce":682284085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6829","nonce":682284085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6833","nonce":682284085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":682284085},{"type":"v1","addr":"172.21.15.37:6831","nonce":682284085}]},"public_addr":"172.21.15.37:6827/682284085","cluster_addr":"172.21.15.37:6829/682284085","heartbeat_back_addr":"172.21.15.37:6833/682284085","heartbeat_front_addr":"172.21.15.37:6831/682284085","state":["exists","up"]},{"osd":10,"uuid":"af8e1072-54de-4e47-9ca4-7b9fdcae474a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6825","nonce":2564518662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6827","nonce":2564518662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6831","nonce":2564518662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2564518662},{"type":"v1","addr":"172.21.15.44:6829","nonce":2564518662}]},"public_addr":"172.21.15.44:6825/2564518662","cluster_addr":"172.21.15.44:6827/2564518662","heartbeat_back_addr":"172.21.15.44:6831/2564518662","heartbeat_front_addr":"172.21.15.44:6829/2564518662","state":["exists","up"]},{"osd":11,"uuid":"359839d6-6c86-46ea-be27-6f5d8fbcdece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6825","nonce":2045769419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6827","nonce":2045769419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6831","nonce":2045769419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2045769419},{"type":"v1","addr":"172.21.15.69:6829","nonce":2045769419}]},"public_addr":"172.21.15.69:6825/2045769419","cluster_addr":"172.21.15.69:6827/2045769419","heartbeat_back_addr":"172.21.15.69:6831/2045769419","heartbeat_front_addr":"172.21.15.69:6829/2045769419","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:18:43.166172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:07.493985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:31.310762+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:19:56.713455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:21.707029+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:20:46.113218+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:11.672805+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:21:36.800408+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:01.265293+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:27.288652+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:22:52.667233+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:23:17.867969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/1479124970":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/4004466629":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:0/774754895":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6801/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:6800/3707854772":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/3175650939":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1473000576":"2024-04-26T13:16:37.834096+0000","172.21.15.37:6801/2107975660":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/755315013":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/1321688497":"2024-04-26T13:17:05.642821+0000","172.21.15.37:0/4026748437":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/2456399257":"2024-04-26T13:16:37.834096+0000","172.21.15.37:0/683134189":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2819711067":"2024-04-26T13:26:11.228097+0000","172.21.15.37:6800/2107975660":"2024-04-26T13:16:37.834096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:36:54.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 msgr2=0x7f795c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 0x7f795c0a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f796c031440 tx=0x7f796c005a20 comp rx=0 tx=0).stop 2024-04-25T13:36:54.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 msgr2=0x7f797019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f796000e9f0 tx=0x7f796000eeb0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 shutdown_connections 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f795c0a3da0 0x7f795c0a6260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7970198590 0x7f7970198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7970106170 0x7f797019e920 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 --2- 172.21.15.37:0/3329774900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79701010a0 0x7f797019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 >> 172.21.15.37:0/3329774900 conn(0x7f7970077f50 msgr2=0x7f79700ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 shutdown_connections 2024-04-25T13:36:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.599+0000 7f797844e640 1 -- 172.21.15.37:0/3329774900 wait complete. 2024-04-25T13:36:54.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/3773344481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4107330 msgr2=0x7fb2f410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/3773344481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4107330 0x7fb2f410ef00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009900 tx=0x7fb2ec02a410 comp rx=0 tx=0).stop 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/3773344481 shutdown_connections 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/3773344481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4107330 0x7fb2f410ef00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/3773344481 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4102c10 0x7fb2f4106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/3773344481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4102260 0x7fb2f4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/3773344481 >> 172.21.15.37:0/3773344481 conn(0x7fb2f40fda80 msgr2=0x7fb2f40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/3773344481 shutdown_connections 2024-04-25T13:36:54.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.713+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/3773344481 wait complete. 2024-04-25T13:36:54.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 Processor -- start 2024-04-25T13:36:54.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 -- start start 2024-04-25T13:36:54.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4107330 0x7fb2f4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 0x7fb2f41a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4112290 con 0x7fb2f4102260 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4112110 con 0x7fb2f4074cd0 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f9dc6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4112410 con 0x7fb2f4107330 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43952/0 (socket says 172.21.15.37:43952) 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f37fe640 1 -- 172.21.15.37:0/111753906 learned_addr learned my addr 172.21.15.37:0/111753906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.714+0000 7fb2f2ffd640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4107330 0x7fb2f4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f3fff640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 0x7fb2f41a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 -- 172.21.15.37:0/111753906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4107330 msgr2=0x7fb2f4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4107330 0x7fb2f4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 -- 172.21.15.37:0/111753906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 msgr2=0x7fb2f41a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 0x7fb2f41a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 -- 172.21.15.37:0/111753906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f41a70a0 con 0x7fb2f4102260 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f3fff640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 0x7fb2f41a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f37fe640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400ebc0 tx=0x7fb2e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4010630 con 0x7fb2f4102260 2024-04-25T13:36:54.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e4010c80 con 0x7fb2f4102260 2024-04-25T13:36:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f41a7390 con 0x7fb2f4102260 2024-04-25T13:36:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4018b60 con 0x7fb2f4102260 2024-04-25T13:36:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.715+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f41a7930 con 0x7fb2f4102260 2024-04-25T13:36:54.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.716+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f4069250 con 0x7fb2f4102260 2024-04-25T13:36:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.717+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2e4018d00 con 0x7fb2f4102260 2024-04-25T13:36:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.718+0000 7fb2f0ff9640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 0x7fb2dc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.718+0000 7fb2f2ffd640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 0x7fb2dc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.718+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fb2e40d0de0 con 0x7fb2f4102260 2024-04-25T13:36:54.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.719+0000 7fb2f2ffd640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 0x7fb2dc0a6290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4075ce0 tx=0x7fb2e8005f90 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:54.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.723+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2e409d450 con 0x7fb2f4102260 2024-04-25T13:36:54.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.946+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb2f410a5c0 con 0x7fb2f4102260 2024-04-25T13:36:54.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.949+0000 7fb2f0ff9640 1 -- 172.21.15.37:0/111753906 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fb2e409cd50 con 0x7fb2f4102260 2024-04-25T13:36:54.949 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:36:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 msgr2=0x7fb2dc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 0x7fb2dc0a6290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4075ce0 tx=0x7fb2e8005f90 comp rx=0 tx=0).stop 2024-04-25T13:36:54.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 msgr2=0x7fb2f4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400ebc0 tx=0x7fb2e400c6a0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 shutdown_connections 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7fb2dc0a3dd0 0x7fb2dc0a6290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2f4074cd0 0x7fb2f41a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb2f4107330 0x7fb2f4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 --2- 172.21.15.37:0/111753906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2f4102260 0x7fb2f4074250 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 >> 172.21.15.37:0/111753906 conn(0x7fb2f40fda80 msgr2=0x7fb2f40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:54.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 shutdown_connections 2024-04-25T13:36:54.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:54.952+0000 7fb2f9dc6640 1 -- 172.21.15.37:0/111753906 wait complete. 2024-04-25T13:36:54.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:36:55.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 -- 172.21.15.37:0/3402399550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 msgr2=0x7f79601035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 --2- 172.21.15.37:0/3402399550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f79601035a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7950009900 tx=0x7f795002a410 comp rx=0 tx=0).stop 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 -- 172.21.15.37:0/3402399550 shutdown_connections 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 --2- 172.21.15.37:0/3402399550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f796010c6d0 0x7f796010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 --2- 172.21.15.37:0/3402399550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7960103b70 0x7f796010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 --2- 172.21.15.37:0/3402399550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f79601035a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.068+0000 7f7964c23640 1 -- 172.21.15.37:0/3402399550 >> 172.21.15.37:0/3402399550 conn(0x7f7960077f70 msgr2=0x7f7960078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.069+0000 7f7964c23640 1 -- 172.21.15.37:0/3402399550 shutdown_connections 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.069+0000 7f7964c23640 1 -- 172.21.15.37:0/3402399550 wait complete. 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.069+0000 7f7964c23640 1 Processor -- start 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.069+0000 7f7964c23640 1 -- start start 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.069+0000 7f7964c23640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796010c6d0 0x7f7960101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:55.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7960101890 0x7f79601a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7960112370 con 0x7f79601031c0 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f79601121f0 con 0x7f796010c6d0 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79601124f0 con 0x7f7960101890 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43962/0 (socket says 172.21.15.37:43962) 2024-04-25T13:36:55.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 -- 172.21.15.37:0/2034032302 learned_addr learned my addr 172.21.15.37:0/2034032302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795dd9b640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796010c6d0 0x7f7960101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795ed9d640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7960101890 0x7f79601a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 -- 172.21.15.37:0/2034032302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7960101890 msgr2=0x7f79601a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7960101890 0x7f79601a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 -- 172.21.15.37:0/2034032302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796010c6d0 msgr2=0x7f7960101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796010c6d0 0x7f7960101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 -- 172.21.15.37:0/2034032302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601a72f0 con 0x7f79601031c0 2024-04-25T13:36:55.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f795e59c640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7950005940 tx=0x7f7950004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795003f030 con 0x7f79601031c0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7950003aa0 con 0x7f79601031c0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601a7580 con 0x7f79601031c0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950035070 con 0x7f79601031c0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.070+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79601a7ac0 con 0x7f79601031c0 2024-04-25T13:36:55.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.071+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7960069250 con 0x7f79601031c0 2024-04-25T13:36:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.072+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7950042030 con 0x7f79601031c0 2024-04-25T13:36:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.073+0000 7f794f7fe640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 0x7f79480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:55.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.073+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8613+0+0 (secure 0 0 0) 0x7f79500f0e20 con 0x7f79601031c0 2024-04-25T13:36:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.073+0000 7f795dd9b640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 0x7f79480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.074+0000 7f795dd9b640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 0x7f79480a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7960102850 tx=0x7f795400a2d0 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:55.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.078+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79500bce10 con 0x7f79601031c0 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/1224505041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3329774900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:36:55.308 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:36:55.309 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mon[32590]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:36:55.309 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: ignoring --setuser ceph since I am not root 2024-04-25T13:36:55.309 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: ignoring --setgroup ceph since I am not root 2024-04-25T13:36:55.309 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-mds[62929]: -- 172.21.15.44:0/3892943151 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602d24d1a40 con 0x5602d24a5000 2024-04-25T13:36:55.309 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:36:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: starting mds.b at 2024-04-25T13:36:55.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.309+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7960107680 con 0x7f79601031c0 2024-04-25T13:36:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a[29200]: 2024-04-25T13:36:54.947+0000 7fefe2c76700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:36:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/1224505041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3329774900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:36:55.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:36:55.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:36:55.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:36:55.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:36:55.352 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:54 smithi037 ceph-mon[29204]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:36:55.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: ignoring --setuser ceph since I am not root 2024-04-25T13:36:55.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: ignoring --setgroup ceph since I am not root 2024-04-25T13:36:55.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mds[63127]: -- 172.21.15.69:0/776319581 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56387058fa40 con 0x563870562800 2024-04-25T13:36:55.385 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: starting mds.i at 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/1224505041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3329774900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T13:36:55.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:36:55.387 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/111753906' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T13:36:55.387 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T13:36:55.387 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:54 smithi069 ceph-mon[32534]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T13:36:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.981+0000 7f794f7fe640 1 -- 172.21.15.37:0/2034032302 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f79500c1110 con 0x7f79601031c0 2024-04-25T13:36:55.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 msgr2=0x7f79480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:55.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 0x7f79480a6260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7960102850 tx=0x7f795400a2d0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 msgr2=0x7f7960100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:55.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7950005940 tx=0x7f7950004270 comp rx=0 tx=0).stop 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 shutdown_connections 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f79480a3da0 0x7f79480a6260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7960101890 0x7f79601a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f796010c6d0 0x7f7960101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 --2- 172.21.15.37:0/2034032302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79601031c0 0x7f7960100e10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 >> 172.21.15.37:0/2034032302 conn(0x7f7960077f70 msgr2=0x7f79600ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 shutdown_connections 2024-04-25T13:36:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:55.985+0000 7f7964c23640 1 -- 172.21.15.37:0/2034032302 wait complete. 2024-04-25T13:36:55.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 -- 172.21.15.37:0/2940349483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b8103ae0 msgr2=0x7f15b810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 --2- 172.21.15.37:0/2940349483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b8103ae0 0x7f15b810bfc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009900 tx=0x7f15a002a480 comp rx=0 tx=0).stop 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 -- 172.21.15.37:0/2940349483 shutdown_connections 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 --2- 172.21.15.37:0/2940349483 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b810c500 0x7f15b810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 --2- 172.21.15.37:0/2940349483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b8103ae0 0x7f15b810bfc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 --2- 172.21.15.37:0/2940349483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b81031c0 0x7f15b81035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 -- 172.21.15.37:0/2940349483 >> 172.21.15.37:0/2940349483 conn(0x7f15b8078000 msgr2=0x7f15b8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 -- 172.21.15.37:0/2940349483 shutdown_connections 2024-04-25T13:36:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.114+0000 7f15be7f0640 1 -- 172.21.15.37:0/2940349483 wait complete. 2024-04-25T13:36:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 Processor -- start 2024-04-25T13:36:56.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 -- start start 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b8103ae0 0x7f15b8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 0x7f15b8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81121c0 con 0x7f15b810c500 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8112040 con 0x7f15b81031c0 2024-04-25T13:36:56.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15be7f0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8112340 con 0x7f15b8103ae0 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15bcd8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 0x7f15b8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.37:57014/0 (socket says 172.21.15.37:57014) 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15b7fff640 1 -- 172.21.15.37:0/480543312 learned_addr learned my addr 172.21.15.37:0/480543312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.115+0000 7f15b77fe640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b8103ae0 0x7f15b8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 -- 172.21.15.37:0/480543312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b8103ae0 msgr2=0x7f15b8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b8103ae0 0x7f15b8101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 -- 172.21.15.37:0/480543312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 msgr2=0x7f15b8100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 -- 172.21.15.37:0/480543312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b81a6ce0 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15b7fff640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15bcd8d640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 0x7f15b8101890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e960 tx=0x7f15b000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0004d20 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b81a6fd0 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15b0005020 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b00116d0 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.116+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81a74a0 con 0x7f15b810c500 2024-04-25T13:36:56.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.117+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b8068e00 con 0x7f15b810c500 2024-04-25T13:36:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.118+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15b00054d0 con 0x7f15b810c500 2024-04-25T13:36:56.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.119+0000 7f15b57fa640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 0x7f15a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.119+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8613+0+0 (secure 0 0 0) 0x7f15b00d0870 con 0x7f15b810c500 2024-04-25T13:36:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.119+0000 7f15b7fff640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 0x7f15a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:56.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.119+0000 7f15b7fff640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 0x7f15a40a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15a8011550 tx=0x7f15a8005f90 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:56.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.124+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15b0016030 con 0x7f15b810c500 2024-04-25T13:36:56.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:55 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:36:56.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.312+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f15b8107680 con 0x7f15b810c500 2024-04-25T13:36:56.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:55 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:36:56.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:55 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T13:36:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.987+0000 7f15b57fa640 1 -- 172.21.15.37:0/480543312 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f15b009c860 con 0x7f15b810c500 2024-04-25T13:36:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:36:56.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 msgr2=0x7f15a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 0x7f15a40a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f15a8011550 tx=0x7f15a8005f90 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 msgr2=0x7f15b8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 0x7f15b8101890 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f15b000e960 tx=0x7f15b000ee20 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 shutdown_connections 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f15a40a3d20 0x7f15a40a61e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15b810c500 0x7f15b8101890 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f15b8103ae0 0x7f15b8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 --2- 172.21.15.37:0/480543312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15b81031c0 0x7f15b8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 >> 172.21.15.37:0/480543312 conn(0x7f15b8078000 msgr2=0x7f15b80ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 shutdown_connections 2024-04-25T13:36:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:56.990+0000 7f15be7f0640 1 -- 172.21.15.37:0/480543312 wait complete. 2024-04-25T13:36:57.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:36:57.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3442358155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 msgr2=0x7f7c74102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3442358155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c74102570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402a410 comp rx=0 tx=0).stop 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3442358155 shutdown_connections 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3442358155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c74107260 0x7f7c7410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3442358155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74102b40 0x7f7c74106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3442358155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c74102570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3442358155 >> 172.21.15.37:0/3442358155 conn(0x7f7c740fda10 msgr2=0x7f7c740ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3442358155 shutdown_connections 2024-04-25T13:36:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.110+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3442358155 wait complete. 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 Processor -- start 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 -- start start 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74107260 0x7f7c7419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 0x7f7c74198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741121f0 con 0x7f7c74102190 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74112070 con 0x7f7c741985d0 2024-04-25T13:36:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c78cbb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7c74112370 con 0x7f7c74107260 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44008/0 (socket says 172.21.15.37:44008) 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.111+0000 7f7c7259c640 1 -- 172.21.15.37:0/3714695691 learned_addr learned my addr 172.21.15.37:0/3714695691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c72d9d640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 0x7f7c74198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c71d9b640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74107260 0x7f7c7419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 -- 172.21.15.37:0/3714695691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74107260 msgr2=0x7f7c7419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74107260 0x7f7c7419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 -- 172.21.15.37:0/3714695691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 msgr2=0x7f7c74198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 0x7f7c74198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 -- 172.21.15.37:0/3714695691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c741990f0 con 0x7f7c74102190 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c72d9d640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 0x7f7c74198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c7259c640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005940 tx=0x7f7c640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6403f030 con 0x7f7c74102190 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c64003cb0 con 0x7f7c74102190 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c74199380 con 0x7f7c74102190 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64035070 con 0x7f7c74102190 2024-04-25T13:36:57.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.112+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741a55d0 con 0x7f7c74102190 2024-04-25T13:36:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.113+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c74068e00 con 0x7f7c74102190 2024-04-25T13:36:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.114+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c64042030 con 0x7f7c74102190 2024-04-25T13:36:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.115+0000 7f7c637fe640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 0x7f7c5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:36:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.115+0000 7f7c71d9b640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 0x7f7c5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:36:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.115+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8059+0+0 (secure 0 0 0) 0x7f7c640f0b90 con 0x7f7c74102190 2024-04-25T13:36:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.115+0000 7f7c71d9b640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 0x7f7c5c0a6260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c68005fd0 tx=0x7f7c68005850 comp rx=0 tx=0).ready entity=mgr.15207 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:36:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.119+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c640bd3b0 con 0x7f7c74102190 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: Cluster is now healthy 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.69:6838/2802356659,v1:172.21.15.69:6839/2802356659] up:boot 2024-04-25T13:36:57.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: mds.? [v2:172.21.15.44:6838/1307958488,v1:172.21.15.44:6839/1307958488] up:boot 2024-04-25T13:36:57.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: fsmap 12 up:standby 2024-04-25T13:36:57.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:36:57.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:36:57.249 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:56 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.311+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7c74199b00 con 0x7f7c74102190 2024-04-25T13:36:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T13:36:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:36:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:36:57.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: Cluster is now healthy 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.69:6838/2802356659,v1:172.21.15.69:6839/2802356659] up:boot 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: mds.? [v2:172.21.15.44:6838/1307958488,v1:172.21.15.44:6839/1307958488] up:boot 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: fsmap 12 up:standby 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:36:57.386 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:56 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: Cluster is now healthy 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/2034032302' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.69:6838/2802356659,v1:172.21.15.69:6839/2802356659] up:boot 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: mds.? [v2:172.21.15.44:6838/1307958488,v1:172.21.15.44:6839/1307958488] up:boot 2024-04-25T13:36:57.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: fsmap 12 up:standby 2024-04-25T13:36:57.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T13:36:57.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: from='mgr.15207 172.21.15.37:0/1803229840' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T13:36:57.391 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:56 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.995+0000 7f7c637fe640 1 -- 172.21.15.37:0/3714695691 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f7c640bccb0 con 0x7f7c74102190 2024-04-25T13:36:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T13:36:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 msgr2=0x7f7c5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 0x7f7c5c0a6260 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c68005fd0 tx=0x7f7c68005850 comp rx=0 tx=0).stop 2024-04-25T13:36:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 msgr2=0x7f7c7419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:36:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7c64005940 tx=0x7f7c640042f0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 shutdown_connections 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:6800/2142450660,v1:172.21.15.37:6801/2142450660] conn(0x7f7c5c0a3da0 0x7f7c5c0a6260 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c741985d0 0x7f7c74198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c74107260 0x7f7c7419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 --2- 172.21.15.37:0/3714695691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c74102190 0x7f7c7419e420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 >> 172.21.15.37:0/3714695691 conn(0x7f7c740fda10 msgr2=0x7f7c740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 shutdown_connections 2024-04-25T13:36:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T13:36:57.998+0000 7f7c78cbb640 1 -- 172.21.15.37:0/3714695691 wait complete. 2024-04-25T13:36:58.016 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:36:58.043 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:36:58.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:36:58.107 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T13:36:58.108 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:36:58.124 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:36:58.141 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T13:36:58.164 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T13:36:58.165 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T13:36:58.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a 2024-04-25T13:36:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:57 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:36:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:57 smithi037 ceph-mon[29204]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:36:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:57 smithi037 ceph-mon[29204]: from='client.? 172.21.15.37:0/3714695691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:58.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 systemd[1]: Stopping Ceph mon.a for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:36:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:57 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:36:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:57 smithi069 ceph-mon[32534]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:36:58.385 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:36:57 smithi069 ceph-mon[32534]: from='client.? 172.21.15.37:0/3714695691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:58.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:57 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/480543312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T13:36:58.389 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:57 smithi044 ceph-mon[32590]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T13:36:58.390 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:57 smithi044 ceph-mon[32590]: from='client.? 172.21.15.37:0/3714695691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T13:36:58.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a[29200]: 2024-04-25T13:36:58.459+0000 7fefe8481700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:36:58.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a[29200]: 2024-04-25T13:36:58.459+0000 7fefe8481700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T13:36:58.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 podman[86394]: 2024-04-25 13:36:58.573117885 +0000 UTC m=+0.221806080 container died d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T13:36:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 podman[86394]: 2024-04-25 13:36:58.951813551 +0000 UTC m=+0.600501739 container cleanup d16148f0b3ad160bf4daf44aaf9fec81641b47e77ae55c027f7cd3bcd22a8f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T13:36:59.101 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 13:36:58 smithi037 bash[86394]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-a 2024-04-25T13:36:59.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T13:36:59.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:36:59.442 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T13:36:59.442 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T13:36:59.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.b 2024-04-25T13:36:59.807 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:59 smithi044 systemd[1]: Stopping Ceph mon.b for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:00.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-b[32586]: 2024-04-25T13:36:59.805+0000 7f819b13a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:00.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-b[32586]: 2024-04-25T13:36:59.805+0000 7f819b13a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:37:00.140 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:36:59 smithi044 podman[68187]: 2024-04-25 13:36:59.980151169 +0000 UTC m=+0.333742128 container died fff21b0440d5ab0ec3621346ebc5a7d2cfe761f98c4953ca2ac0aa337364bee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:37:00.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:37:00 smithi044 podman[68187]: 2024-04-25 13:37:00.447953496 +0000 UTC m=+0.801544457 container cleanup fff21b0440d5ab0ec3621346ebc5a7d2cfe761f98c4953ca2ac0aa337364bee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:37:00.640 INFO:journalctl@ceph.mon.b.smithi044.stdout:Apr 25 13:37:00 smithi044 bash[68187]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-b 2024-04-25T13:37:00.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:37:00.532+0000 7f5273758700 -1 client.15240: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T13:37:00.851 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:37:00.534+0000 7f5273758700 -1 client.15240: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T13:37:00.883 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T13:37:00.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:00.932 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T13:37:00.933 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T13:37:00.933 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.c 2024-04-25T13:37:01.254 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:00 smithi069 systemd[1]: Stopping Ceph mon.c for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:01.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-c[32530]: 2024-04-25T13:37:01.253+0000 7fa6b8389700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:01.635 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-c[32530]: 2024-04-25T13:37:01.253+0000 7fa6b8389700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T13:37:01.636 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:01 smithi069 podman[68410]: 2024-04-25 13:37:01.448309035 +0000 UTC m=+0.313872378 container died 98847380f4eb88fd247b50e96993d81fcb6080833b7d0adb6d157c3e57049977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:37:02.080 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:01 smithi069 podman[68410]: 2024-04-25 13:37:01.879183622 +0000 UTC m=+0.744746977 container cleanup 98847380f4eb88fd247b50e96993d81fcb6080833b7d0adb6d157c3e57049977 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:37:02.080 INFO:journalctl@ceph.mon.c.smithi069.stdout:Apr 25 13:37:01 smithi069 bash[68410]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mon-c 2024-04-25T13:37:02.301 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T13:37:02.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:02.351 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T13:37:02.352 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T13:37:02.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x 2024-04-25T13:37:02.683 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:02 smithi037 systemd[1]: Stopping Ceph mgr.x for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:02.998 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:37:02.682+0000 7f530e010700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:02.998 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x[29438]: 2024-04-25T13:37:02.682+0000 7f530e010700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T13:37:02.998 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:02 smithi037 podman[86515]: 2024-04-25 13:37:02.714832664 +0000 UTC m=+0.153498286 container died 4773bce144a732077bf3e4a50fc76decdd0c976aea1960e17c12d6b8ac191789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:03.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:03 smithi037 podman[86515]: 2024-04-25 13:37:03.106423152 +0000 UTC m=+0.545088774 container cleanup 4773bce144a732077bf3e4a50fc76decdd0c976aea1960e17c12d6b8ac191789 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T13:37:03.351 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 13:37:03 smithi037 bash[86515]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-x 2024-04-25T13:37:03.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T13:37:03.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:03.586 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T13:37:03.589 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T13:37:03.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.y 2024-04-25T13:37:03.889 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:37:03 smithi044 systemd[1]: Stopping Ceph mgr.y for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:04.204 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:37:03 smithi044 podman[68311]: 2024-04-25 13:37:03.916083729 +0000 UTC m=+0.129772782 container died 554bc2fe87fc8bbf90a535002db03879b458b96d9e49fcb77f7554656607435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:04.541 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:37:04 smithi044 podman[68311]: 2024-04-25 13:37:04.304241635 +0000 UTC m=+0.517930699 container cleanup 554bc2fe87fc8bbf90a535002db03879b458b96d9e49fcb77f7554656607435e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:04.541 INFO:journalctl@ceph.mgr.y.smithi044.stdout:Apr 25 13:37:04 smithi044 bash[68311]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-y 2024-04-25T13:37:04.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T13:37:04.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:04.716 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T13:37:04.716 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T13:37:04.716 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.z 2024-04-25T13:37:05.054 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:37:04 smithi069 systemd[1]: Stopping Ceph mgr.z for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:05.325 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:37:05 smithi069 podman[68531]: 2024-04-25 13:37:05.054332713 +0000 UTC m=+0.144730693 container died 2344106ab1fe9d539e5a97bea742af9fb0c68197b978b4d3f6a5332436ec34ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:05.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:37:05 smithi069 podman[68531]: 2024-04-25 13:37:05.408204479 +0000 UTC m=+0.498602464 container cleanup 2344106ab1fe9d539e5a97bea742af9fb0c68197b978b4d3f6a5332436ec34ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T13:37:05.635 INFO:journalctl@ceph.mgr.z.smithi069.stdout:Apr 25 13:37:05 smithi069 bash[68531]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mgr-z 2024-04-25T13:37:05.830 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T13:37:05.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:05.862 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T13:37:05.863 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T13:37:05.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.0 2024-04-25T13:37:06.177 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:05 smithi037 systemd[1]: Stopping Ceph osd.0 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:06.601 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0[44049]: 2024-04-25T13:37:06.177+0000 7fc1ad825700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:06.601 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0[44049]: 2024-04-25T13:37:06.177+0000 7fc1ad825700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-25T13:37:06.601 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0[44049]: 2024-04-25T13:37:06.177+0000 7fc1ad825700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:11.820 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:11 smithi037 podman[86637]: 2024-04-25 13:37:11.522968822 +0000 UTC m=+5.456318973 container died c5279eb86b415e8c17f47b238743b31449ce704a8abe8719d962c71c97fada15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T13:37:12.101 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:11 smithi037 podman[86637]: 2024-04-25 13:37:11.919359132 +0000 UTC m=+5.852709283 container cleanup c5279eb86b415e8c17f47b238743b31449ce704a8abe8719d962c71c97fada15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T13:37:12.101 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:11 smithi037 bash[86637]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0 2024-04-25T13:37:12.546 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86655]: 2024-04-25 13:37:12.17906369 +0000 UTC m=+0.658608926 container remove c5279eb86b415e8c17f47b238743b31449ce704a8abe8719d962c71c97fada15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T13:37:12.851 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86744]: 2024-04-25 13:37:12.450278669 +0000 UTC m=+0.018674689 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:12.851 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86744]: 2024-04-25 13:37:12.563029486 +0000 UTC m=+0.131425495 container create eaaa0007abf5ec895bea4c303565e191bca5ce37f45065367636fd3fa48d613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:37:12.851 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86744]: 2024-04-25 13:37:12.790434527 +0000 UTC m=+0.358830537 container init eaaa0007abf5ec895bea4c303565e191bca5ce37f45065367636fd3fa48d613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T13:37:12.851 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86744]: 2024-04-25 13:37:12.793871916 +0000 UTC m=+0.362267936 container start eaaa0007abf5ec895bea4c303565e191bca5ce37f45065367636fd3fa48d613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T13:37:13.351 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:12 smithi037 podman[86744]: 2024-04-25 13:37:12.915700437 +0000 UTC m=+0.484096454 container attach eaaa0007abf5ec895bea4c303565e191bca5ce37f45065367636fd3fa48d613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T13:37:13.352 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 13:37:13 smithi037 podman[86744]: 2024-04-25 13:37:13.10094251 +0000 UTC m=+0.669338527 container died eaaa0007abf5ec895bea4c303565e191bca5ce37f45065367636fd3fa48d613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:13.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T13:37:13.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:13.737 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T13:37:13.737 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T13:37:13.737 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.1 2024-04-25T13:37:14.061 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:13 smithi044 systemd[1]: Stopping Ceph osd.1 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:14.389 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1[37184]: 2024-04-25T13:37:14.060+0000 7f50f4c72700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:14.390 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1[37184]: 2024-04-25T13:37:14.060+0000 7f50f4c72700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-25T13:37:14.390 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1[37184]: 2024-04-25T13:37:14.060+0000 7f50f4c72700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:19.795 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:19 smithi044 podman[68437]: 2024-04-25 13:37:19.495562818 +0000 UTC m=+5.556935913 container died b17c45403b9b6d098ac1ca5b62a5945ce9177b5c4ccb07df0ce222e0de66b4a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T13:37:20.140 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68437]: 2024-04-25 13:37:20.003007211 +0000 UTC m=+6.064380311 container cleanup b17c45403b9b6d098ac1ca5b62a5945ce9177b5c4ccb07df0ce222e0de66b4a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T13:37:20.140 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 bash[68437]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1 2024-04-25T13:37:20.587 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68456]: 2024-04-25 13:37:20.262998032 +0000 UTC m=+0.757826583 container remove b17c45403b9b6d098ac1ca5b62a5945ce9177b5c4ccb07df0ce222e0de66b4a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T13:37:20.890 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68545]: 2024-04-25 13:37:20.489957993 +0000 UTC m=+0.017961154 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:20.890 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68545]: 2024-04-25 13:37:20.613895127 +0000 UTC m=+0.141898314 container create f8df17bc22dc9a230d69aab1e4abc3ea93ae1ba370bfcf18e33da4fb502ef716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T13:37:20.890 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68545]: 2024-04-25 13:37:20.833201543 +0000 UTC m=+0.361204689 container init f8df17bc22dc9a230d69aab1e4abc3ea93ae1ba370bfcf18e33da4fb502ef716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T13:37:20.890 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68545]: 2024-04-25 13:37:20.836368775 +0000 UTC m=+0.364371923 container start f8df17bc22dc9a230d69aab1e4abc3ea93ae1ba370bfcf18e33da4fb502ef716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:37:21.141 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:20 smithi044 podman[68545]: 2024-04-25 13:37:20.933543509 +0000 UTC m=+0.461546662 container attach f8df17bc22dc9a230d69aab1e4abc3ea93ae1ba370bfcf18e33da4fb502ef716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T13:37:21.428 INFO:journalctl@ceph.osd.1.smithi044.stdout:Apr 25 13:37:21 smithi044 podman[68545]: 2024-04-25 13:37:21.142738528 +0000 UTC m=+0.670741676 container died f8df17bc22dc9a230d69aab1e4abc3ea93ae1ba370bfcf18e33da4fb502ef716 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:37:21.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T13:37:21.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:21.909 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T13:37:21.909 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T13:37:21.909 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.2 2024-04-25T13:37:22.203 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:21 smithi069 systemd[1]: Stopping Ceph osd.2 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:22.462 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2[37663]: 2024-04-25T13:37:22.202+0000 7f54f0b50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:22.462 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2[37663]: 2024-04-25T13:37:22.202+0000 7f54f0b50700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-25T13:37:22.462 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2[37663]: 2024-04-25T13:37:22.202+0000 7f54f0b50700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:27.804 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:27 smithi069 podman[68656]: 2024-04-25 13:37:27.520543467 +0000 UTC m=+5.419328605 container died 354de7c521c8986d29383c9e6aaedae183593103aa94dc5d9837c6a5e9c677a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T13:37:28.135 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:27 smithi069 podman[68656]: 2024-04-25 13:37:27.894884067 +0000 UTC m=+5.793669199 container cleanup 354de7c521c8986d29383c9e6aaedae183593103aa94dc5d9837c6a5e9c677a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T13:37:28.135 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:27 smithi069 bash[68656]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2 2024-04-25T13:37:28.514 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68686]: 2024-04-25 13:37:28.170919729 +0000 UTC m=+0.644514621 container remove 354de7c521c8986d29383c9e6aaedae183593103aa94dc5d9837c6a5e9c677a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:37:28.832 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68778]: 2024-04-25 13:37:28.417253392 +0000 UTC m=+0.018061134 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:28.832 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68778]: 2024-04-25 13:37:28.538683742 +0000 UTC m=+0.139491462 container create 55a08f9029c514c45f81dc6a952f730b0eea88c407629bc8ab157c01d8230bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T13:37:28.832 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68778]: 2024-04-25 13:37:28.749849624 +0000 UTC m=+0.350657354 container init 55a08f9029c514c45f81dc6a952f730b0eea88c407629bc8ab157c01d8230bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:28.832 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68778]: 2024-04-25 13:37:28.753145422 +0000 UTC m=+0.353953149 container start 55a08f9029c514c45f81dc6a952f730b0eea88c407629bc8ab157c01d8230bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T13:37:28.833 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:28 smithi069 podman[68778]: 2024-04-25 13:37:28.832736536 +0000 UTC m=+0.433544262 container attach 55a08f9029c514c45f81dc6a952f730b0eea88c407629bc8ab157c01d8230bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T13:37:29.135 INFO:journalctl@ceph.osd.2.smithi069.stdout:Apr 25 13:37:29 smithi069 podman[68778]: 2024-04-25 13:37:29.043677087 +0000 UTC m=+0.644484811 container died 55a08f9029c514c45f81dc6a952f730b0eea88c407629bc8ab157c01d8230bb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:37:29.657 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T13:37:29.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:29.695 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T13:37:29.695 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T13:37:29.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.3 2024-04-25T13:37:30.022 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:29 smithi037 systemd[1]: Stopping Ceph osd.3 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:30.294 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:37:30.021+0000 7f3a73559700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:30.295 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:37:30.021+0000 7f3a73559700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-25T13:37:30.295 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:30 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:37:30.021+0000 7f3a73559700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:33.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:33.470+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:34.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:33 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:33.673+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:34.351 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:33 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:37:33.884+0000 7f3a6bb45700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:11.131781+0000 front 2024-04-25T13:37:11.131831+0000 (oldest deadline 2024-04-25T13:37:33.431998+0000) 2024-04-25T13:37:34.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:34 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:34.004+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:34.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:34 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:34.450+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:34.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:34.440+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:34.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:34.537+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:35.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:34 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:34.697+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:35.351 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:34 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T13:37:34.853+0000 7f3a6bb45700 -1 osd.3 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:11.131781+0000 front 2024-04-25T13:37:11.131831+0000 (oldest deadline 2024-04-25T13:37:33.431998+0000) 2024-04-25T13:37:35.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:35.012+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:35.806 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:35 smithi037 podman[86855]: 2024-04-25 13:37:35.516629209 +0000 UTC m=+5.614629652 container died 2451519dd283bc3f176e207e2531fc2bb049b552dfbe9395d1eb9435929e8c4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T13:37:35.806 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:35 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:35.411+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:35.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:35.412+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:35.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:35.551+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:36.101 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:36.030+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:36.101 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:35 smithi037 podman[86855]: 2024-04-25 13:37:35.905255034 +0000 UTC m=+6.003255476 container cleanup 2451519dd283bc3f176e207e2531fc2bb049b552dfbe9395d1eb9435929e8c4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:36.101 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:35 smithi037 bash[86855]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3 2024-04-25T13:37:36.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:35 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:35.714+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:36.587 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:36 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:36.361+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:36.587 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86885]: 2024-04-25 13:37:36.156723677 +0000 UTC m=+0.637244577 container remove 2451519dd283bc3f176e207e2531fc2bb049b552dfbe9395d1eb9435929e8c4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T13:37:36.851 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86975]: 2024-04-25 13:37:36.490560481 +0000 UTC m=+0.018570493 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:36.851 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86975]: 2024-04-25 13:37:36.607417003 +0000 UTC m=+0.135427004 container create 87d6a7a5f430e4186ab4224fcf35f46f255bce5d819899f962a39091664db8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T13:37:36.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:36.391+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:36.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:36.553+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:37.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:36 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:36.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:37.165 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:37 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:37.022+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:37.165 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86975]: 2024-04-25 13:37:36.859746468 +0000 UTC m=+0.387756468 container init 87d6a7a5f430e4186ab4224fcf35f46f255bce5d819899f962a39091664db8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:37.165 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86975]: 2024-04-25 13:37:36.863247494 +0000 UTC m=+0.391257500 container start 87d6a7a5f430e4186ab4224fcf35f46f255bce5d819899f962a39091664db8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T13:37:37.165 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:36 smithi037 podman[86975]: 2024-04-25 13:37:36.943011604 +0000 UTC m=+0.471021608 container attach 87d6a7a5f430e4186ab4224fcf35f46f255bce5d819899f962a39091664db8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:37:37.512 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 13:37:37 smithi037 podman[86975]: 2024-04-25 13:37:37.166198995 +0000 UTC m=+0.694208996 container died 87d6a7a5f430e4186ab4224fcf35f46f255bce5d819899f962a39091664db8ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T13:37:37.512 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:37 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:37.325+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:37.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:37.574+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:37.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:37.344+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:37.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T13:37:37.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:37.831 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T13:37:37.831 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T13:37:37.832 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.4 2024-04-25T13:37:38.101 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:37 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:37.980+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:38.139 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:37 smithi044 systemd[1]: Stopping Ceph osd.4 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:38.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:37 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:37.734+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:38.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:38.304+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:38.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:38.532+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:38.639 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4[44083]: 2024-04-25T13:37:38.140+0000 7fa10ef9c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:38.639 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4[44083]: 2024-04-25T13:37:38.140+0000 7fa10ef9c700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-25T13:37:38.640 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4[44083]: 2024-04-25T13:37:38.140+0000 7fa10ef9c700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:38.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:38 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:38.369+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:39.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:38 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:38.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:39.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:39 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:39.002+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:39.385 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:39 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:39.341+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:39.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:39 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:39.361+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:39.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:39 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:39.505+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:40.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:39 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:39.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:40.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:39 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:39.996+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:40.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:40 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:40.358+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:40.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:40 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:40.512+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:40.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:40 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:40.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:40.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:40 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:40.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:41.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:40 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:40.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:41.325 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:40 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:40.982+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:41.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:41.325+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:41.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:41.325+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:41.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:41 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:41.533+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:41.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:41 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:41.335+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:41.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:41 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:41.335+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:41.636 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:41 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:41.415+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:41.889 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:41 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:41.768+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:41.890 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:41 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:41.627+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:42.330 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:41.993+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:42.581 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:42 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:42.301+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:42.582 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:42 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:42.301+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:42.582 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:42 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:42.438+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:42.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:42.330+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:42.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:42.330+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:42.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:42 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:42.582+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:42.889 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:42 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:42.584+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:42.889 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:42 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:42.765+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:43.283 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:42 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:42.996+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:43.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:43 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:43.283+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:43.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:43 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:43.283+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:43.622 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:43 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:43.448+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:43.622 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:43 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:43.342+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:43.622 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:43 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:43.342+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:43.742 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:43 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:43.554+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:43.743 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:43 smithi044 podman[68676]: 2024-04-25 13:37:43.473382943 +0000 UTC m=+5.432244098 container died 06c74657362f24e75aa0ac795f3ddbaa3dec46e93dbb19cf7756c268ff639f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:43.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:43 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:43.621+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:43.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:43 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:43.621+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:44.117 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:43 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:43.741+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:44.117 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:43 smithi044 podman[68676]: 2024-04-25 13:37:43.841155587 +0000 UTC m=+5.800016722 container cleanup 06c74657362f24e75aa0ac795f3ddbaa3dec46e93dbb19cf7756c268ff639f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T13:37:44.118 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:43 smithi044 bash[68676]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4 2024-04-25T13:37:44.249 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:43 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:43.978+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:44.249 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:43 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:43.978+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:44.389 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68691]: 2024-04-25 13:37:44.117721311 +0000 UTC m=+0.642017006 container remove 06c74657362f24e75aa0ac795f3ddbaa3dec46e93dbb19cf7756c268ff639f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:44.577 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:44 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:44.320+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:44.577 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:44 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:44.320+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:44.577 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:44 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:44.452+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:44.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:44.249+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:44.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:44.249+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:44 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:44.540+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.319189448 +0000 UTC m=+0.018503856 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.418595427 +0000 UTC m=+0.117909864 container create 288fe2965255201b0c17862591da1c5d8d9eca7665fc9d4b3181203a3ebf1112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.637631139 +0000 UTC m=+0.336945531 container init 288fe2965255201b0c17862591da1c5d8d9eca7665fc9d4b3181203a3ebf1112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.640751929 +0000 UTC m=+0.340066325 container start 288fe2965255201b0c17862591da1c5d8d9eca7665fc9d4b3181203a3ebf1112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:37:44.737 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.737670576 +0000 UTC m=+0.436984983 container attach 288fe2965255201b0c17862591da1c5d8d9eca7665fc9d4b3181203a3ebf1112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:37:44.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:44 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:44.577+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:44.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:44 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:44.577+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:45.140 INFO:journalctl@ceph.osd.4.smithi044.stdout:Apr 25 13:37:44 smithi044 podman[68787]: 2024-04-25 13:37:44.9438537 +0000 UTC m=+0.643168101 container died 288fe2965255201b0c17862591da1c5d8d9eca7665fc9d4b3181203a3ebf1112 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T13:37:45.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:44 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:44.756+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:45.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:44 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:44.756+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:45.294 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:44.956+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:45.294 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:44.956+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:45.552 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T13:37:45.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:45.595 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T13:37:45.596 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T13:37:45.596 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.5 2024-04-25T13:37:45.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:45 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:45.293+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:45.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:45 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:45.293+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:45.604 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:45.463+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:45.604 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:45.338+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:45.604 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:45.338+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:45.879 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 systemd[1]: Stopping Ceph osd.5 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:45.880 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:45.604+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:45.880 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:45.604+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:45.889 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:45 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:45.538+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:45.889 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:45 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:45.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:45.889 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:45 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:45.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:46.134 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:45.879+0000 7faeed4d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:46.135 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:45.879+0000 7faeed4d9700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-25T13:37:46.135 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:45 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:45.879+0000 7faeed4d9700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:46.303 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:45 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:45.918+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:46.303 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:45 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:45.918+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:46.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:46 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:46.303+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:46.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:46 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:46.303+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:46.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:46.426+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:46.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:46.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:46.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:46.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:46.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:46.385+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:46.636 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:46 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:46.385+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:46.802 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:46 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:46.507+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:47.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:46 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:46.802+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:47.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:46 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:46.802+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:47.237 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:46 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:46.913+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:47.237 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:46 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:46.913+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:47.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:47 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:47.327+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:47.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:47 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:47.327+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:47.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:47 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:47.393+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:47.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:47 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:47.375+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:47.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:47 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:47.375+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:47.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:47 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:47.602+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:47.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:47 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:47.602+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:47.802 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:47 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:47.469+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:48.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:47 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:47.801+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:48.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:47 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:47.801+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:48.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:47 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:47.801+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:48.350 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:47 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:47.932+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:48.350 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:47 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:47.932+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:48.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:48 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:48.350+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:48.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:48 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:48.350+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:48.777 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:48 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:48.466+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:48.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:48.416+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:48.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:48.416+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:48.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:48.416+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:48.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:48.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:48.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:48.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:48.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:48 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:48.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:49.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:48 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:48.777+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:49.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:48 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:48.777+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:49.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:48 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:48.777+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:49.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:48 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:48.929+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:49.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:48 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:48.929+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:49.738 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:49 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:49.487+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:49.850 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:49.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:49.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:49.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:49.392+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:49.392+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:49.440+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:49.440+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:49.589+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:49.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:49.589+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:49.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:49 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:49.589+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:50.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:49 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:49.737+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:50.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:49 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:49.737+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:50.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:49 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:49.737+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:50.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:49.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:50.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:49.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:50.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:49.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:50.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:50.355+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.467519+0000 front 2024-04-25T13:37:10.467585+0000 (oldest deadline 2024-04-25T13:37:33.967523+0000) 2024-04-25T13:37:50.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:50.355+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.969216+0000 front 2024-04-25T13:37:17.969432+0000 (oldest deadline 2024-04-25T13:37:40.869246+0000) 2024-04-25T13:37:50.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5[44639]: 2024-04-25T13:37:50.355+0000 7faee9acd700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:24.970124+0000 front 2024-04-25T13:37:24.970173+0000 (oldest deadline 2024-04-25T13:37:49.670223+0000) 2024-04-25T13:37:50.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:50.425+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:50.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:50.425+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:50.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:50.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:50.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:50.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:50.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:50 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:50.563+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:50.743 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:50 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:50.438+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:50.850 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:50.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:50.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:50.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:50.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:50.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:51.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:50 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:50.743+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:51.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:50 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:50.743+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:51.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:50 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:50.743+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:51.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:50.904+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:51.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:50.904+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:51.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:50 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:50.904+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:51.560 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:51 smithi069 podman[68885]: 2024-04-25 13:37:51.337931031 +0000 UTC m=+5.552393826 container died 93717c685db80b46409e55b678bfcfce2f1283866a88115699cc4efdd8809665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T13:37:51.560 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:51 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:51.384+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:51.561 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:51 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:51.384+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:51.630 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:51 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:51.558+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:51.630 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:51 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:51.559+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:51.630 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:51 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:51.559+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:51.697 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:51 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:51.410+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:51.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:51.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:51.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:51.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:51.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:51.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:51.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:51 smithi069 podman[68885]: 2024-04-25 13:37:51.779819738 +0000 UTC m=+5.994282532 container cleanup 93717c685db80b46409e55b678bfcfce2f1283866a88115699cc4efdd8809665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:37:51.885 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:51 smithi069 bash[68885]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5 2024-04-25T13:37:52.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:51 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:51.697+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:52.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:51 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:51.697+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:52.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:51 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:51.697+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:52.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:51.876+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:52.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:51.876+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:52.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:51 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:51.876+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:52.382 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:51 smithi069 podman[68902]: 2024-04-25 13:37:51.980916861 +0000 UTC m=+0.635684880 container remove 93717c685db80b46409e55b678bfcfce2f1283866a88115699cc4efdd8809665 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T13:37:52.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:52 smithi069 podman[68993]: 2024-04-25 13:37:52.295345501 +0000 UTC m=+0.018293114 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:37:52.635 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:52 smithi069 podman[68993]: 2024-04-25 13:37:52.398188423 +0000 UTC m=+0.121136004 container create 868a6d99da611f797d45048d2fa914e0353db8130a6bbebeaaa01278006df3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:52.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:52.382+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:52.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:52.382+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:52.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:52.565+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:52.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:52.565+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:52.636 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:52 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:52.565+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:52.672 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:52 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:52.389+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:52.672 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:52 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:52.389+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:52.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:52.379+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:52.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:52.379+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:52.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:52.379+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:53.006 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:52 smithi069 podman[68993]: 2024-04-25 13:37:52.717176937 +0000 UTC m=+0.440124529 container init 868a6d99da611f797d45048d2fa914e0353db8130a6bbebeaaa01278006df3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:37:53.007 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:52 smithi069 podman[68993]: 2024-04-25 13:37:52.720288554 +0000 UTC m=+0.443236136 container start 868a6d99da611f797d45048d2fa914e0353db8130a6bbebeaaa01278006df3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:53.007 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:52 smithi069 podman[68993]: 2024-04-25 13:37:52.817156602 +0000 UTC m=+0.540104196 container attach 868a6d99da611f797d45048d2fa914e0353db8130a6bbebeaaa01278006df3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:37:53.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:52 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:52.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:53.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:52 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:52.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:53.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:52 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:52.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:53.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:52.914+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:53.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:52.914+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:53.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:52 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:52.914+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:53.352 INFO:journalctl@ceph.osd.5.smithi069.stdout:Apr 25 13:37:53 smithi069 podman[68993]: 2024-04-25 13:37:53.007475743 +0000 UTC m=+0.730423336 container died 868a6d99da611f797d45048d2fa914e0353db8130a6bbebeaaa01278006df3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:37:53.610 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:53 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:53.422+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:53.610 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:53 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:53.422+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:53.633 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T13:37:53.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:53 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:53.378+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:53.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:53 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:53.378+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:53.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:37:53.713 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T13:37:53.713 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T13:37:53.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.6 2024-04-25T13:37:53.740 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:53.353+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:53.740 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:53.353+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:53.740 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:53.353+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:53.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:53 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:53.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:53.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:53 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:53.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:53.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:53 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:53.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:54.032 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:53 smithi037 systemd[1]: Stopping Ceph osd.6 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:37:54.032 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:53.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:54.032 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:53.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:54.032 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:53 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:53.950+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:54.033 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.032+0000 7f7afb24c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:37:54.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:53 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:53.669+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:54.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:53 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:53.669+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:54.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:53 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:53.669+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:54.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.032+0000 7f7afb24c700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-25T13:37:54.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.032+0000 7f7afb24c700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:37:54.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:54.374+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:54.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:54.374+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:54.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:54.387+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:54.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:54.387+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:54.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:54.387+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:54.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:54.445+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:54.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:54.445+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:54.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:54.619+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:54.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:54.619+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:54.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:54 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:54.619+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:55.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:54.720+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:55.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:54.720+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:55.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:54 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:54.720+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:55.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.945+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:55.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.945+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:55.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:54.945+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:55.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:55.362+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:55.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:55.362+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:55.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:55.405+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:55.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:55.405+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:55.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:55.405+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:55.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:55 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:55.396+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:55.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:55 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:55.396+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:55.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:55 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:55.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:55.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:55 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:55.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:55.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:55 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:55.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:56.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:55.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:56.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:55.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:56.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:55 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:55.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:56.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:55.995+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:56.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:55.995+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:56.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:55 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:55.995+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:56.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:56.327+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:56.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:56.327+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:56.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:56.327+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:37:56.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:56 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:56.431+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:56.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:56 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:56.431+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:56.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:56 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:56.431+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:56.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:56.389+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:56.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:56.389+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:56.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:56.389+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:37:56.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:56.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:56.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:56.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:56.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:56 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:56.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:57.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:56.677+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:57.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:56.677+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:57.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:56.677+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:57.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:56 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:56.677+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:37:57.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:57.034+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:57.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:57.034+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:57.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:57.034+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:57.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:57.588+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:57.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:57.588+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:57.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:57.588+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:57.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:57.373+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:57.636 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:57.373+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:57.636 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:57 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:57.373+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:37:57.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:57.296+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:57.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:57.296+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:57.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:57.296+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:37:57.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:57.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:57.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:57.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:57.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:57 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:57.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:58.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:57.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:58.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:57.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:58.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:57.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:58.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:57 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:57.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:37:58.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.008+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:58.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.008+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:58.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.008+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:58.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.008+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:33.005583+0000 front 2024-04-25T13:37:33.005601+0000 (oldest deadline 2024-04-25T13:37:57.105672+0000) 2024-04-25T13:37:58.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:58.378+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:58.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:58.378+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:58.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:58.378+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:37:58.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:58.328+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:58.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:58.328+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:58.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:58.328+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:37:58.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:58.451+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:58.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:58.451+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:58.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:58.451+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:37:59.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:58.635+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:59.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:58.635+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:59.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:58 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:58.635+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:59.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:58.690+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:37:59.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:58.690+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:37:59.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:58.690+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:37:59.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:58.690+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:37:59.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.967+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:07.401260+0000 front 2024-04-25T13:37:07.401280+0000 (oldest deadline 2024-04-25T13:37:33.301426+0000) 2024-04-25T13:37:59.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.967+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.902995+0000 front 2024-04-25T13:37:17.903057+0000 (oldest deadline 2024-04-25T13:37:43.202928+0000) 2024-04-25T13:37:59.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.967+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.604607+0000 front 2024-04-25T13:37:26.604623+0000 (oldest deadline 2024-04-25T13:37:49.504541+0000) 2024-04-25T13:37:59.351 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:58 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6[57618]: 2024-04-25T13:37:58.967+0000 7f7af3838700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:33.005583+0000 front 2024-04-25T13:37:33.005601+0000 (oldest deadline 2024-04-25T13:37:57.105672+0000) 2024-04-25T13:37:59.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:59.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:37:59.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:59.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:37:59.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:37:59.610+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:37:59.635 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:59.382+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:37:59.636 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:59.382+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:37:59.636 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:37:59 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:37:59.382+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:37:59.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:59.351+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:37:59.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:59.351+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:37:59.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:37:59.351+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:37:59.707 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:59 smithi037 podman[87083]: 2024-04-25 13:37:59.434813153 +0000 UTC m=+5.521894105 container died 10d3cf795b91bd07a05fcc4b880bba551f85cbe1945cacc2027cd31474abc4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:37:59.708 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:59 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:59.444+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:37:59.708 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:59 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:59.444+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:37:59.708 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:37:59 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:37:59.444+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:00.099 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:59 smithi037 podman[87083]: 2024-04-25 13:37:59.807212999 +0000 UTC m=+5.894293948 container cleanup 10d3cf795b91bd07a05fcc4b880bba551f85cbe1945cacc2027cd31474abc4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:00.100 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:37:59 smithi037 bash[87083]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6 2024-04-25T13:38:00.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:59.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:00.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:59.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:00.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:59.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:00.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:37:59 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:37:59.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:00.350 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87100]: 2024-04-25 13:38:00.100182395 +0000 UTC m=+0.662356450 container remove 10d3cf795b91bd07a05fcc4b880bba551f85cbe1945cacc2027cd31474abc4bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T13:38:00.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:00.359+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:00.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:00.359+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:00.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:00.359+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:00.656 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.342944851 +0000 UTC m=+0.018468469 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.459347831 +0000 UTC m=+0.134871483 container create c299ac70796e4cc589fb12974772d27f56dde6cdad478202165e011c99e5b8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.653517828 +0000 UTC m=+0.329041438 container init c299ac70796e4cc589fb12974772d27f56dde6cdad478202165e011c99e5b8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.656990155 +0000 UTC m=+0.332513765 container start c299ac70796e4cc589fb12974772d27f56dde6cdad478202165e011c99e5b8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:00.402+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:00.402+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:00.657 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:00.402+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:00.658 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:00 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:00.402+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:00.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:00.427+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:00.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:00.427+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:00.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:00.427+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:00.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:00.584+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:00.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:00.584+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:00.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:00 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:00.584+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:00.957 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.753545985 +0000 UTC m=+0.429069602 container attach c299ac70796e4cc589fb12974772d27f56dde6cdad478202165e011c99e5b8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True) 2024-04-25T13:38:01.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:00.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:01.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:00.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:01.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:00.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:01.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:00 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:00.670+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:01.248 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 13:38:00 smithi037 podman[87191]: 2024-04-25 13:38:00.958023325 +0000 UTC m=+0.633546938 container died c299ac70796e4cc589fb12974772d27f56dde6cdad478202165e011c99e5b8c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T13:38:01.427 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.340+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:01.427 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.340+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:01.427 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.340+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:01.539 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:01.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:01.539 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:01.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:01.540 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:01.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:01.540 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:01 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:01.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:01.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T13:38:01.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:01.648 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T13:38:01.648 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T13:38:01.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.7 2024-04-25T13:38:01.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:01.447+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:01.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:01.447+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:01.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:01.447+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:01.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:01.602+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:01.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:01.602+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:01.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:01 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:01.602+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:01.985 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 systemd[1]: Stopping Ceph osd.7 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:01.986 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:01.712+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:01.986 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:01.712+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:01.986 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:01.712+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:01.986 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:01.712+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.985+0000 7fea5dca9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.985+0000 7fea5dca9700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:01 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:01.985+0000 7fea5dca9700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:02.293+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:02.293+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:02.293 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:02.293+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:02.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:02.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:02.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:02.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:02.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:02.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:02.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:02 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:02.363+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:02.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:02.425+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:02.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:02.425+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:02.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:02.425+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:02.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:02.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:02.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:02.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:02.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:02 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:02.631+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:03.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:02.752+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:03.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:02.752+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:03.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:02.752+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:03.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:02 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:02.752+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:03.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:03.349+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:03.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:03.349+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:03.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:03.349+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:03.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:03 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:03.349+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:03.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:03.318+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:03.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:03.318+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:03.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:03.318+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:03.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:03.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:03.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:03.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:03.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:03.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:03.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:03.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:03.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:03.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:03.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:03 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:03.609+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:04.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:03.792+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:04.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:03.792+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:04.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:03.792+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:04.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:03.792+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:04.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:04.339+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:04.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:04.339+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:04.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:04.339+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:04.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:04.359+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:04.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:04.359+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:04.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:04.359+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:04.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:04 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:04.359+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:04.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:04.450+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:04.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:04.450+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:04.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:04.450+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:04.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:04.574+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:04.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:04.574+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:04.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:04 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:04.574+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:05.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:04.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:05.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:04.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:05.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:04.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:05.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:04.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:05.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:04 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:04.774+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:05.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:05.328+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:05.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:05.328+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:05.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:05.328+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:05.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:05 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:05.328+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:05.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:05.321+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:05.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:05.321+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:05.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:05.321+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:05.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:05.441+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:05.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:05.441+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:05.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:05.441+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:05.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:05.441+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:05.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:05.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:05.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:05.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:05.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:05.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:05.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:05 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:05.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:06.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:05.744+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:06.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:05.744+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:06.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:05.744+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:06.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:05.744+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:06.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:05 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:05.744+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:06.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:06.284+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.903846+0000 front 2024-04-25T13:37:16.903937+0000 (oldest deadline 2024-04-25T13:37:41.003960+0000) 2024-04-25T13:38:06.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:06.284+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.904868+0000 front 2024-04-25T13:37:26.904922+0000 (oldest deadline 2024-04-25T13:37:52.205048+0000) 2024-04-25T13:38:06.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:06.284+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:32.205740+0000 front 2024-04-25T13:37:32.205796+0000 (oldest deadline 2024-04-25T13:37:56.305467+0000) 2024-04-25T13:38:06.640 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7[50919]: 2024-04-25T13:38:06.284+0000 7fea56295700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.206422+0000 front 2024-04-25T13:37:42.206400+0000 (oldest deadline 2024-04-25T13:38:05.706616+0000) 2024-04-25T13:38:06.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:06.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:06.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:06.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:06.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:06.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:06.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:06 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:06.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:06.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:06.474+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:06.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:06.474+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:06.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:06.474+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:06.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:06.474+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:06.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:06.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:06.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:06.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:06.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:06.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:06.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:06 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:06.617+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:07.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:06.781+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:07.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:06.781+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:07.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:06.781+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:07.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:06.781+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:07.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:06 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:06.781+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:07.639 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:07 smithi044 podman[68901]: 2024-04-25 13:38:07.358975129 +0000 UTC m=+5.477272437 container died 936c3a2f65b902737df124a9078526658f6d7673db62790d578d2adb684d22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T13:38:07.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:07.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:07.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:07.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:07.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:07.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:07.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:07 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:07.362+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:07.509+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:07.509+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:07.509+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:07.509+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:07.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:07.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:07.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:07.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:07.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:07.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:07 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:07.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:08.139 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:07 smithi044 podman[68901]: 2024-04-25 13:38:07.813797383 +0000 UTC m=+5.932094682 container cleanup 936c3a2f65b902737df124a9078526658f6d7673db62790d578d2adb684d22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:08.139 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:07 smithi044 bash[68901]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7 2024-04-25T13:38:08.139 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[68921]: 2024-04-25 13:38:08.012426228 +0000 UTC m=+0.654586763 container remove 936c3a2f65b902737df124a9078526658f6d7673db62790d578d2adb684d22a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:38:08.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:07.808+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:08.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:07.808+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:08.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:07.808+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:08.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:07.808+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:08.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:07 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:07.808+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:08.763 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[69018]: 2024-04-25 13:38:08.346830381 +0000 UTC m=+0.018105590 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[69018]: 2024-04-25 13:38:08.455126427 +0000 UTC m=+0.126401658 container create d70f9342f4c4fcf0c591c8da055247aed9cf13412468596620904c1140bf679e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:08.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:08.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:08.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:08.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:08.764 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:08.763+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:08.850 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:08.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:08.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:08.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:08.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:08.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:08.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:08 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:08.389+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:08.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:08.473+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:08.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:08.473+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:08.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:08.473+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:08.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:08.473+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:08.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:08.630+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:08.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:08.630+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:08.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:08.630+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:08.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:08 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:08.630+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:09.106 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[69018]: 2024-04-25 13:38:08.800279504 +0000 UTC m=+0.471554694 container init d70f9342f4c4fcf0c591c8da055247aed9cf13412468596620904c1140bf679e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T13:38:09.106 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[69018]: 2024-04-25 13:38:08.803580558 +0000 UTC m=+0.474855748 container start d70f9342f4c4fcf0c591c8da055247aed9cf13412468596620904c1140bf679e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:38:09.106 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:08 smithi044 podman[69018]: 2024-04-25 13:38:08.91635786 +0000 UTC m=+0.587633058 container attach d70f9342f4c4fcf0c591c8da055247aed9cf13412468596620904c1140bf679e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T13:38:09.389 INFO:journalctl@ceph.osd.7.smithi044.stdout:Apr 25 13:38:09 smithi044 podman[69018]: 2024-04-25 13:38:09.106432565 +0000 UTC m=+0.777707756 container died d70f9342f4c4fcf0c591c8da055247aed9cf13412468596620904c1140bf679e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T13:38:09.690 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T13:38:09.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:09.730 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T13:38:09.731 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T13:38:09.731 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.8 2024-04-25T13:38:09.756 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:09.470+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:09.756 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:09.470+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:09.756 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:09.470+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:09.757 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:09.470+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:09.757 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:09.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:09.757 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:09.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:09.757 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:09.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:09.757 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:09 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:09.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:09.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:09.416+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:09.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:09.416+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:09.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:09.416+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:09.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:09 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:09.416+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:09.889 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:09.758+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:09.890 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:09.758+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:09.890 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:09.758+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:09.890 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:09.758+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:09.890 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:09 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:09.758+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:10.031 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:09 smithi069 systemd[1]: Stopping Ceph osd.8 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:10.385 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.030+0000 7f3608cc1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:10.385 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.030+0000 7f3608cc1700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-25T13:38:10.385 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.030+0000 7f3608cc1700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:38:10.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:10.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:10.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:10.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:10.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:10.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:10.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:10 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:10.441+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.507+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.507+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.507+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:10.507+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:10.550+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:10.550+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:10.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:10.550+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:10.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:10 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:10.550+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:11.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:10.746+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:11.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:10.746+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:11.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:10.746+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:11.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:10.746+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:11.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:10 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:10.746+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:11.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:11.481+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:11.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:11.481+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:11.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:11.481+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:11.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:11 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:11.481+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:11.478+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:11.478+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:11.478+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:11.478+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:11.596+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:11.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:11.596+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:11.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:11.596+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:11.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:11 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:11.596+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:12.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:11 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:11.778+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:12.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:11 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:11.778+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:12.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:11 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:11.778+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:12.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:11 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:11.778+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:12.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:11 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:11.778+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:12.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:12.464+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:12.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:12.464+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:12.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:12.464+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:12.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:12 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:12.464+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:12.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:12.556+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:12.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:12.556+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:12.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:12.556+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:12.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:12.556+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:12.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:12.443+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:12.886 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:12.443+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:12.886 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:12.443+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:12.886 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:12 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:12.443+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:13.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:12.785+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:13.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:12.785+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:13.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:12.785+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:13.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:12.785+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:13.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:12.785+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:13.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:13.478+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:13.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:13.478+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:13.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:13.478+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:13.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:13.478+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:13.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:13 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:13.478+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:13.430+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:13.430+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:13.430+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:13.430+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:13.570+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:13.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:13.570+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:13.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:13.570+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:13.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:13.570+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:13.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:13 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:13.570+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:14.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:13 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:13.769+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:14.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:13 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:13.769+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:14.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:13 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:13.769+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:14.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:13 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:13.769+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:14.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:13 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:13.769+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:14.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:14 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:14.488+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:14.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:14 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:14.488+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:14.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:14 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:14.488+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:14.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:14 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:14.488+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:14.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:14 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:14.488+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:14.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.735226+0000 front 2024-04-25T13:37:18.735199+0000 (oldest deadline 2024-04-25T13:37:40.936040+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:14.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.737578+0000 front 2024-04-25T13:37:26.737620+0000 (oldest deadline 2024-04-25T13:37:49.037748+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:14.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:31.938824+0000 front 2024-04-25T13:37:31.938732+0000 (oldest deadline 2024-04-25T13:37:56.038756+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8[51200]: 2024-04-25T13:38:14.472+0000 7f36012ad700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:42.341189+0000 front 2024-04-25T13:37:42.340985+0000 (oldest deadline 2024-04-25T13:38:04.641069+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:14.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:14.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:14.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:14.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:14.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:14.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:14.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:14.886 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:14 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:14.586+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:15.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:15.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:15.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:15.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:15.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:15.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:14 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:14.814+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:15.822 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:15 smithi069 podman[69100]: 2024-04-25 13:38:15.570497147 +0000 UTC m=+5.645484517 container died 5c69cacbe80c746a51a136166948f43e16be067f1446f6542542b8e98698b96d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:38:15.822 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:15.594+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:15.822 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:15.594+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:15.823 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:15.594+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:15.823 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:15.594+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:15.823 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:15.594+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:15.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:15 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:15.500+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:15.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:15 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:15.500+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:15.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:15 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:15.500+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:15.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:15 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:15.500+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:15.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:15 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:15.500+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:16.105 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:15 smithi069 podman[69100]: 2024-04-25 13:38:15.913561141 +0000 UTC m=+5.988548584 container cleanup 5c69cacbe80c746a51a136166948f43e16be067f1446f6542542b8e98698b96d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T13:38:16.105 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:15 smithi069 bash[69100]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8 2024-04-25T13:38:16.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:16.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:16.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:16.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:16.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:16.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:15 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:15.767+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:16.385 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69117]: 2024-04-25 13:38:16.106141121 +0000 UTC m=+0.536811990 container remove 5c69cacbe80c746a51a136166948f43e16be067f1446f6542542b8e98698b96d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:38:16.825 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:16 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:16.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:16.825 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:16 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:16.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:16.825 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:16 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:16.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:16.826 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:16 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:16.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:16.826 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:16 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:16.593+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:16.826 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69211]: 2024-04-25 13:38:16.424334757 +0000 UTC m=+0.017936689 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:16.826 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69211]: 2024-04-25 13:38:16.531826386 +0000 UTC m=+0.125428314 container create dacfb28819028b4913a780d01b252f11e11e416da2864f96ff09be8b6f125c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T13:38:16.827 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69211]: 2024-04-25 13:38:16.742648154 +0000 UTC m=+0.336250082 container init dacfb28819028b4913a780d01b252f11e11e416da2864f96ff09be8b6f125c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T13:38:16.827 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69211]: 2024-04-25 13:38:16.745668586 +0000 UTC m=+0.339270516 container start dacfb28819028b4913a780d01b252f11e11e416da2864f96ff09be8b6f125c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True) 2024-04-25T13:38:16.827 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:16 smithi069 podman[69211]: 2024-04-25 13:38:16.825820964 +0000 UTC m=+0.419422894 container attach dacfb28819028b4913a780d01b252f11e11e416da2864f96ff09be8b6f125c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T13:38:16.850 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:16.465+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:16.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:16.465+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:16.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:16.465+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:16.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:16.465+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:16.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:16 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:16.465+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:17.135 INFO:journalctl@ceph.osd.8.smithi069.stdout:Apr 25 13:38:17 smithi069 podman[69211]: 2024-04-25 13:38:17.032183857 +0000 UTC m=+0.625785789 container died dacfb28819028b4913a780d01b252f11e11e416da2864f96ff09be8b6f125c3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T13:38:17.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:17.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:17.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:17.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:17.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:17.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:16 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:16.721+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:17.712 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T13:38:17.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:17.789 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T13:38:17.789 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T13:38:17.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.9 2024-04-25T13:38:17.816 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:17.418+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:17.816 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:17.418+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:17.816 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:17.418+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:17.816 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:17.418+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:17.817 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:17.418+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:17.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:17 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:17.629+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:17.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:17 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:17.629+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:17.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:17 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:17.629+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:17.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:17 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:17.629+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:17.885 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:17 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:17.629+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:18.100 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:17 smithi037 systemd[1]: Stopping Ceph osd.9 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:18.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:18.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:18.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:18.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:18.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:18.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:17 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:17.681+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:18.351 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.100+0000 7f72e4b42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:18.351 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.100+0000 7f72e4b42700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-25T13:38:18.351 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.100+0000 7f72e4b42700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:38:18.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.403+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:18.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.403+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:18.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.403+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:18.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.403+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:18.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:18 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:18.403+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:19.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:18.673+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:19.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:18.673+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:19.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:18.673+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:19.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:18.673+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:19.138 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:18 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:18.673+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:19.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:19.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:19.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:19.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:19.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:19.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:18 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:18.672+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:19.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:19.386+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:19.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:19.386+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:19.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:19.386+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:19.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:19.386+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:19.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:19 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:19.386+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:20.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:19.676+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:20.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:19.676+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:20.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:19.676+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:20.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:19.676+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:20.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:19 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:19.676+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:20.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:20.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:20.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:20.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:20.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:20.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:19 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:19.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:20.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:20 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:20.392+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:54.529307+0000 front 2024-04-25T13:37:54.529345+0000 (oldest deadline 2024-04-25T13:38:19.829454+0000) 2024-04-25T13:38:21.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:20.698+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:21.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:20.698+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:21.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:20.698+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:21.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:20.698+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:21.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:20 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:20.698+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:21.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.649+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:21.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:20 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:20.650+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:21.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:21 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:21.412+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:54.529307+0000 front 2024-04-25T13:37:54.529345+0000 (oldest deadline 2024-04-25T13:38:19.829454+0000) 2024-04-25T13:38:22.036 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:22.036 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:22.036 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:22.036 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:22.036 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:22.037 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:21.683+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:22.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:22.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:22.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:22.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:22.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:22.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:22.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:21 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:21.655+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:08.621034+0000 front 2024-04-25T13:37:08.621377+0000 (oldest deadline 2024-04-25T13:37:32.721148+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:16.722996+0000 front 2024-04-25T13:37:16.722973+0000 (oldest deadline 2024-04-25T13:37:40.222961+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:26.024700+0000 front 2024-04-25T13:37:26.024792+0000 (oldest deadline 2024-04-25T13:37:49.524754+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:29.525182+0000 front 2024-04-25T13:37:29.525144+0000 (oldest deadline 2024-04-25T13:38:00.125904+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:49.328475+0000 front 2024-04-25T13:37:49.328654+0000 (oldest deadline 2024-04-25T13:38:12.828500+0000) 2024-04-25T13:38:22.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:22 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9[64239]: 2024-04-25T13:38:22.420+0000 7f72dd12e700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:54.529307+0000 front 2024-04-25T13:37:54.529345+0000 (oldest deadline 2024-04-25T13:38:19.829454+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:23.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:22 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:22.733+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:23.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:23.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:23.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:23.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:23.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:23.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:23.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:22 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:22.696+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:23.851 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:23 smithi037 podman[87296]: 2024-04-25 13:38:23.578688159 +0000 UTC m=+5.583574290 container died b5e0a3a4177353cd51132bcc13a979dd1ad23e98dbb353490b37944c174b49a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-25T13:38:24.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:24.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:24.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:24.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:24.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:24.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:23 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:23.689+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:24.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:24.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:24.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:24.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:24.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:24.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:24.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:23 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:23.699+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:24.308 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:24 smithi037 podman[87296]: 2024-04-25 13:38:24.041512415 +0000 UTC m=+6.046398537 container cleanup b5e0a3a4177353cd51132bcc13a979dd1ad23e98dbb353490b37944c174b49a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T13:38:24.308 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:24 smithi037 bash[87296]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9 2024-04-25T13:38:24.600 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:24 smithi037 podman[87311]: 2024-04-25 13:38:24.309395582 +0000 UTC m=+0.727140344 container remove b5e0a3a4177353cd51132bcc13a979dd1ad23e98dbb353490b37944c174b49a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-25T13:38:25.004 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:24 smithi037 podman[87404]: 2024-04-25 13:38:24.538851401 +0000 UTC m=+0.019509791 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:25.004 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:24 smithi037 podman[87404]: 2024-04-25 13:38:24.651823664 +0000 UTC m=+0.132482041 container create c0d29dbb283df88456187e7d77f92f23ac3e6e0e75cec1b4e13a2db63e85672c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:25.004 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:25 smithi037 podman[87404]: 2024-04-25 13:38:25.004451578 +0000 UTC m=+0.485109965 container init c0d29dbb283df88456187e7d77f92f23ac3e6e0e75cec1b4e13a2db63e85672c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T13:38:25.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:25.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:25.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:25.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:25.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:25.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:24 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:24.688+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:25.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:25.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:25.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:25.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:25.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:25.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:25.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:24 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:24.748+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:25.282 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:25 smithi037 podman[87404]: 2024-04-25 13:38:25.008128906 +0000 UTC m=+0.488787294 container start c0d29dbb283df88456187e7d77f92f23ac3e6e0e75cec1b4e13a2db63e85672c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:25.283 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:25 smithi037 podman[87404]: 2024-04-25 13:38:25.095677579 +0000 UTC m=+0.576335965 container attach c0d29dbb283df88456187e7d77f92f23ac3e6e0e75cec1b4e13a2db63e85672c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T13:38:25.601 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 13:38:25 smithi037 podman[87404]: 2024-04-25 13:38:25.324059526 +0000 UTC m=+0.804717901 container died c0d29dbb283df88456187e7d77f92f23ac3e6e0e75cec1b4e13a2db63e85672c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:38:26.020 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T13:38:26.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:26.098 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T13:38:26.098 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T13:38:26.099 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.10 2024-04-25T13:38:26.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:26.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:26.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:26.125 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:26.126 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:26.126 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:26.126 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:25 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:25.757+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:26.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:25 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:25.737+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:26.389 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 systemd[1]: Stopping Ceph osd.10 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.421+0000 7fc29c698700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.421+0000 7fc29c698700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.421+0000 7fc29c698700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:26.727 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:26.728 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:26.728 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:27.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:27.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:27.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:27.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:27.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:27.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:26.709+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:27.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:27.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:26 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:26.727+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:28.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:28.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:28.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:28.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:28.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:28.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:27 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:27.748+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:28.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:28.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:28.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:28.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:28.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:28.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:28.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:27 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:27.686+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:29.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:28 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:28.774+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:29.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:29.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:29.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:29.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:29.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:29.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:29.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:28 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:28.718+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:30.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:29 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:29.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:30.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:30.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:30.139 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:30.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:30.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:30.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:30.140 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:29 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:29.738+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:31.041 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.998186+0000 front 2024-04-25T13:37:10.998101+0000 (oldest deadline 2024-04-25T13:37:33.298257+0000) 2024-04-25T13:38:31.041 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:18.499762+0000 front 2024-04-25T13:37:18.499731+0000 (oldest deadline 2024-04-25T13:37:43.799925+0000) 2024-04-25T13:38:31.041 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:23.800315+0000 front 2024-04-25T13:37:23.800419+0000 (oldest deadline 2024-04-25T13:37:47.800824+0000) 2024-04-25T13:38:31.041 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T13:37:34.603780+0000 front 2024-04-25T13:37:34.603860+0000 (oldest deadline 2024-04-25T13:37:56.303567+0000) 2024-04-25T13:38:31.042 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:39.204656+0000 front 2024-04-25T13:37:39.204638+0000 (oldest deadline 2024-04-25T13:38:04.504690+0000) 2024-04-25T13:38:31.042 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.005980+0000 front 2024-04-25T13:37:48.006002+0000 (oldest deadline 2024-04-25T13:38:13.905826+0000) 2024-04-25T13:38:31.042 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:30 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10[57654]: 2024-04-25T13:38:30.750+0000 7fc294c84700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:58.908521+0000 front 2024-04-25T13:37:58.908540+0000 (oldest deadline 2024-04-25T13:38:20.008381+0000) 2024-04-25T13:38:31.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:31.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:31.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:31.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:31.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:31.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:30 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:30.814+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:32.039 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:31 smithi044 podman[69121]: 2024-04-25 13:38:31.73884036 +0000 UTC m=+5.432959109 container died a53122d54c3f9801f436597a7cb72764713de48dde5bf6902dce070b980b9889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T13:38:32.088 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:32.089 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:32.089 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:32.089 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:32.089 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:32.089 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:31.803+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:32.364 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69121]: 2024-04-25 13:38:32.130618129 +0000 UTC m=+5.824736877 container cleanup a53122d54c3f9801f436597a7cb72764713de48dde5bf6902dce070b980b9889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:32.364 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 bash[69121]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10 2024-04-25T13:38:32.640 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69139]: 2024-04-25 13:38:32.365022893 +0000 UTC m=+0.617885880 container remove a53122d54c3f9801f436597a7cb72764713de48dde5bf6902dce070b980b9889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:38:33.026 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69232]: 2024-04-25 13:38:32.592097612 +0000 UTC m=+0.017952302 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:33.026 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69232]: 2024-04-25 13:38:32.699243454 +0000 UTC m=+0.125098132 container create 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T13:38:33.026 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69232]: 2024-04-25 13:38:32.92667348 +0000 UTC m=+0.352528158 container init 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T13:38:33.026 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:32 smithi044 podman[69232]: 2024-04-25 13:38:32.929869637 +0000 UTC m=+0.355724321 container start 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T13:38:33.026 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:33 smithi044 podman[69232]: 2024-04-25 13:38:33.026710753 +0000 UTC m=+0.452565431 container attach 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:33.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:32 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:32.839+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:33.390 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:33 smithi044 podman[69232]: 2024-04-25 13:38:33.232921838 +0000 UTC m=+0.658776517 container died 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:38:33.793 INFO:journalctl@ceph.osd.10.smithi044.stdout:Apr 25 13:38:33 smithi044 podman[69258]: 2024-04-25 13:38:33.77209133 +0000 UTC m=+0.533590129 container remove 7515f099bb12a67a9a39040375ac728586c7ddfa72bc274f4c10eb655a103294 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T13:38:33.798 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T13:38:33.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:33.878 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T13:38:33.878 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T13:38:33.878 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.11 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:34.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:34.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:33.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:34.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:33 smithi069 systemd[1]: Stopping Ceph osd.11 for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:34.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.172+0000 7fe174deb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:34.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.172+0000 7fe174deb700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-25T13:38:34.635 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.172+0000 7fe174deb700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T13:38:35.385 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:35.385 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:35.385 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:35.385 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:35.386 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:35.386 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:35.386 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:34 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:34.895+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:36.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:36.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:36.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:36.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:36.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:36.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:36.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:35 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:35.857+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:37.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:37.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:37.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:37.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:37.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.850+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:37.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:37.137 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:36 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:36.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:38.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:38.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:38.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:38.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:38.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:38.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:38.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.854+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:38.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:37 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:37.855+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6820 osd.8 since back 2024-04-25T13:38:12.766666+0000 front 2024-04-25T13:38:12.766683+0000 (oldest deadline 2024-04-25T13:38:36.866659+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T13:37:10.456942+0000 front 2024-04-25T13:37:10.456823+0000 (oldest deadline 2024-04-25T13:37:33.956879+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6804 osd.1 since back 2024-04-25T13:37:17.859347+0000 front 2024-04-25T13:37:17.859251+0000 (oldest deadline 2024-04-25T13:37:43.159274+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6804 osd.2 since back 2024-04-25T13:37:25.460000+0000 front 2024-04-25T13:37:25.460102+0000 (oldest deadline 2024-04-25T13:37:48.360179+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6812 osd.4 since back 2024-04-25T13:37:41.163021+0000 front 2024-04-25T13:37:41.162976+0000 (oldest deadline 2024-04-25T13:38:04.662918+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6812 osd.5 since back 2024-04-25T13:37:48.163933+0000 front 2024-04-25T13:37:48.163889+0000 (oldest deadline 2024-04-25T13:38:13.464044+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T13:37:56.964972+0000 front 2024-04-25T13:37:56.965010+0000 (oldest deadline 2024-04-25T13:38:21.064837+0000) 2024-04-25T13:38:39.136 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.44:6820 osd.7 since back 2024-04-25T13:38:06.965885+0000 front 2024-04-25T13:38:06.965825+0000 (oldest deadline 2024-04-25T13:38:32.265766+0000) 2024-04-25T13:38:39.137 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:38 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11[57769]: 2024-04-25T13:38:38.851+0000 7fe16d3d7700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.69:6820 osd.8 since back 2024-04-25T13:38:12.766666+0000 front 2024-04-25T13:38:12.766683+0000 (oldest deadline 2024-04-25T13:38:36.866659+0000) 2024-04-25T13:38:39.765 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:39 smithi069 podman[69329]: 2024-04-25 13:38:39.507536571 +0000 UTC m=+5.437173956 container died b3baf1d3614abec92c61274d6caabd19e20085d3d23c80be6bf2e5a3ec8f2bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T13:38:40.131 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:39 smithi069 podman[69329]: 2024-04-25 13:38:39.863818935 +0000 UTC m=+5.793456314 container cleanup b3baf1d3614abec92c61274d6caabd19e20085d3d23c80be6bf2e5a3ec8f2bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T13:38:40.131 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:39 smithi069 bash[69329]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11 2024-04-25T13:38:40.384 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69346]: 2024-04-25 13:38:40.131544464 +0000 UTC m=+0.622896150 container remove b3baf1d3614abec92c61274d6caabd19e20085d3d23c80be6bf2e5a3ec8f2bf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T13:38:40.767 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69437]: 2024-04-25 13:38:40.381534175 +0000 UTC m=+0.018079017 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T13:38:40.767 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69437]: 2024-04-25 13:38:40.482397204 +0000 UTC m=+0.118942078 container create 37a2dc7dda40cf77d432f3c79baa846130a7c78874258429e1b9d3801c6d36c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T13:38:40.767 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69437]: 2024-04-25 13:38:40.709505037 +0000 UTC m=+0.346049868 container init 37a2dc7dda40cf77d432f3c79baa846130a7c78874258429e1b9d3801c6d36c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:40.768 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69437]: 2024-04-25 13:38:40.71240705 +0000 UTC m=+0.348951878 container start 37a2dc7dda40cf77d432f3c79baa846130a7c78874258429e1b9d3801c6d36c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T13:38:40.768 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:40 smithi069 podman[69437]: 2024-04-25 13:38:40.767790559 +0000 UTC m=+0.404335407 container attach 37a2dc7dda40cf77d432f3c79baa846130a7c78874258429e1b9d3801c6d36c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T13:38:41.135 INFO:journalctl@ceph.osd.11.smithi069.stdout:Apr 25 13:38:41 smithi069 podman[69437]: 2024-04-25 13:38:41.003033021 +0000 UTC m=+0.639577853 container died 37a2dc7dda40cf77d432f3c79baa846130a7c78874258429e1b9d3801c6d36c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:38:41.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T13:38:41.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:41.605 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T13:38:41.606 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T13:38:41.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.a 2024-04-25T13:38:41.910 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:41 smithi037 systemd[1]: Stopping Ceph mds.a for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:42.351 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a[71095]: 2024-04-25T13:38:41.909+0000 7f2f029a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:42.351 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:41 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a[71095]: 2024-04-25T13:38:41.909+0000 7f2f029a0700 -1 mds.a *** got signal Terminated *** 2024-04-25T13:38:44.009 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:43 smithi037 podman[87532]: 2024-04-25 13:38:43.737333815 +0000 UTC m=+1.922706703 container died 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T13:38:44.350 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:44 smithi037 podman[87532]: 2024-04-25 13:38:44.109556174 +0000 UTC m=+2.294929129 container cleanup 9c8e8392d2e13ee5507483e84986c5635b65a21cf3ed7623d84f41e5c959bcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T13:38:44.351 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 13:38:44 smithi037 bash[87532]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-a 2024-04-25T13:38:44.583 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T13:38:44.618 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:44.618 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T13:38:44.618 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T13:38:44.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.d 2024-04-25T13:38:44.954 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:44 smithi037 systemd[1]: Stopping Ceph mds.d for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:45.351 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d[70145]: 2024-04-25T13:38:44.954+0000 7fa2640ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:45.351 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:44 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d[70145]: 2024-04-25T13:38:44.954+0000 7fa2640ed700 -1 mds.d *** got signal Terminated *** 2024-04-25T13:38:48.351 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:48 smithi037 podman[87652]: 2024-04-25 13:38:48.031701993 +0000 UTC m=+3.186023365 container died e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T13:38:48.755 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:48 smithi037 podman[87652]: 2024-04-25 13:38:48.487903884 +0000 UTC m=+3.642225273 container cleanup e06315133780d84481cec5977bee9dc396b7b6d542dfecf5471e65f302af1bb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T13:38:48.755 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 13:38:48 smithi037 bash[87652]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-d 2024-04-25T13:38:48.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T13:38:48.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:48.957 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T13:38:48.957 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T13:38:48.957 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.g 2024-04-25T13:38:49.301 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:49 smithi037 systemd[1]: Stopping Ceph mds.g for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:49.601 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g[70746]: 2024-04-25T13:38:49.300+0000 7faf6a5e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:49.601 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:49 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g[70746]: 2024-04-25T13:38:49.300+0000 7faf6a5e5700 -1 mds.g *** got signal Terminated *** 2024-04-25T13:38:53.341 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:52 smithi037 podman[87774]: 2024-04-25 13:38:52.981264267 +0000 UTC m=+3.793961945 container died 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:38:53.601 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:53 smithi037 podman[87774]: 2024-04-25 13:38:53.441579293 +0000 UTC m=+4.254276979 container cleanup 630ddbb5eadfa83299436c02d795d1e3144ce68069641b86c45213a21eea4e5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T13:38:53.601 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 13:38:53 smithi037 bash[87774]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-g 2024-04-25T13:38:53.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T13:38:53.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:53.912 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T13:38:53.912 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T13:38:53.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.j 2024-04-25T13:38:54.261 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:54 smithi037 systemd[1]: Stopping Ceph mds.j for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:54.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j[69776]: 2024-04-25T13:38:54.261+0000 7fe94dcf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:54.601 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:54 smithi037 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j[69776]: 2024-04-25T13:38:54.261+0000 7fe94dcf7700 -1 mds.j *** got signal Terminated *** 2024-04-25T13:38:57.753 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:57 smithi037 podman[87895]: 2024-04-25 13:38:57.361101253 +0000 UTC m=+3.215991422 container died 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T13:38:58.095 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:57 smithi037 podman[87895]: 2024-04-25 13:38:57.836726332 +0000 UTC m=+3.691616509 container cleanup 305be8b65f17b5539ea5adb7b5e000ca06a3e0b90e495dca5dc047574ba86a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:38:58.095 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 13:38:57 smithi037 bash[87895]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-j 2024-04-25T13:38:58.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T13:38:58.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:38:58.296 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T13:38:58.296 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T13:38:58.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.b 2024-04-25T13:38:58.606 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:38:58 smithi044 systemd[1]: Stopping Ceph mds.b for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:38:58.889 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:38:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: 2024-04-25T13:38:58.606+0000 7ff727d9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:38:58.889 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:38:58 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b[62925]: 2024-04-25T13:38:58.606+0000 7ff727d9b700 -1 mds.b *** got signal Terminated *** 2024-04-25T13:39:02.881 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:39:02 smithi044 podman[69336]: 2024-04-25 13:39:02.622006976 +0000 UTC m=+4.117734184 container died 55d02c9377c39ce6046fae9f45795efd6127beb6da7d470591a5c561bc8e8f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T13:39:03.139 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:39:02 smithi044 podman[69336]: 2024-04-25 13:39:02.98108244 +0000 UTC m=+4.476809638 container cleanup 55d02c9377c39ce6046fae9f45795efd6127beb6da7d470591a5c561bc8e8f30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:03.140 INFO:journalctl@ceph.mds.b.smithi044.stdout:Apr 25 13:39:02 smithi044 bash[69336]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-b 2024-04-25T13:39:03.410 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T13:39:03.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:03.445 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T13:39:03.445 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T13:39:03.445 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.e 2024-04-25T13:39:03.639 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:03 smithi044 systemd[1]: Stopping Ceph mds.e for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:04.139 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e[62297]: 2024-04-25T13:39:03.760+0000 7f98c0bd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:04.139 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:03 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e[62297]: 2024-04-25T13:39:03.760+0000 7f98c0bd0700 -1 mds.e *** got signal Terminated *** 2024-04-25T13:39:07.139 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:06 smithi044 podman[69455]: 2024-04-25 13:39:06.879790998 +0000 UTC m=+3.216360864 container died 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T13:39:07.594 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:07 smithi044 podman[69455]: 2024-04-25 13:39:07.368828888 +0000 UTC m=+3.705398735 container cleanup 600bdc8f5abecb12e22952f012420f50d19e8c29ce0407c819af78a03a465e55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T13:39:07.595 INFO:journalctl@ceph.mds.e.smithi044.stdout:Apr 25 13:39:07 smithi044 bash[69455]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-e 2024-04-25T13:39:07.727 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T13:39:07.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:07.761 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T13:39:07.761 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T13:39:07.761 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.h 2024-04-25T13:39:07.889 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:07 smithi044 systemd[1]: Stopping Ceph mds.h for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:08.389 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h[62615]: 2024-04-25T13:39:08.105+0000 7f58fbd18700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:08.389 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:08 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h[62615]: 2024-04-25T13:39:08.105+0000 7f58fbd18700 -1 mds.h *** got signal Terminated *** 2024-04-25T13:39:11.639 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:11 smithi044 podman[69587]: 2024-04-25 13:39:11.367951004 +0000 UTC m=+3.386559878 container died 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:39:11.989 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:11 smithi044 podman[69587]: 2024-04-25 13:39:11.813947214 +0000 UTC m=+3.832556081 container cleanup 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:39:11.989 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:11 smithi044 bash[69587]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h 2024-04-25T13:39:12.249 INFO:journalctl@ceph.mds.h.smithi044.stdout:Apr 25 13:39:11 smithi044 podman[69601]: 2024-04-25 13:39:11.990050882 +0000 UTC m=+0.617775101 container remove 98d1ca477f87329be386ba987a80a21e5ed92146703c61faddc7800b45369a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T13:39:12.253 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T13:39:12.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:12.286 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T13:39:12.286 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T13:39:12.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.k 2024-04-25T13:39:12.617 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:12 smithi044 systemd[1]: Stopping Ceph mds.k for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:12.889 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k[61978]: 2024-04-25T13:39:12.617+0000 7fbf706d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:12.889 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:12 smithi044 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k[61978]: 2024-04-25T13:39:12.617+0000 7fbf706d0700 -1 mds.k *** got signal Terminated *** 2024-04-25T13:39:14.876 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:14 smithi044 podman[69711]: 2024-04-25 13:39:14.619003367 +0000 UTC m=+2.111625507 container died c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T13:39:15.139 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:15 smithi044 podman[69711]: 2024-04-25 13:39:15.076031084 +0000 UTC m=+2.568653224 container cleanup c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T13:39:15.139 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:15 smithi044 bash[69711]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k 2024-04-25T13:39:15.498 INFO:journalctl@ceph.mds.k.smithi044.stdout:Apr 25 13:39:15 smithi044 podman[69725]: 2024-04-25 13:39:15.368534279 +0000 UTC m=+0.748737495 container remove c26924caa949b9b734266429467f54b2414b0148ddd4c0d9e26dcdf3dce3e293 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T13:39:15.519 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T13:39:15.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:15.552 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T13:39:15.552 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T13:39:15.552 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.c 2024-04-25T13:39:15.853 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:15 smithi069 systemd[1]: Stopping Ceph mds.c for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:16.111 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c[62815]: 2024-04-25T13:39:15.853+0000 7f9fa536c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:16.111 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:15 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c[62815]: 2024-04-25T13:39:15.853+0000 7f9fa536c700 -1 mds.c *** got signal Terminated *** 2024-04-25T13:39:20.135 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:19 smithi069 podman[69541]: 2024-04-25 13:39:19.860193014 +0000 UTC m=+4.111984824 container died 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T13:39:20.496 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:20 smithi069 podman[69541]: 2024-04-25 13:39:20.304130291 +0000 UTC m=+4.555922098 container cleanup 7502ab2f929778ad8bf5832f208783ac88a35888912d82819ea1c0aa47393be7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T13:39:20.496 INFO:journalctl@ceph.mds.c.smithi069.stdout:Apr 25 13:39:20 smithi069 bash[69541]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-c 2024-04-25T13:39:20.722 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T13:39:20.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:20.756 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T13:39:20.757 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T13:39:20.757 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.f 2024-04-25T13:39:21.083 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:20 smithi069 systemd[1]: Stopping Ceph mds.f for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:21.385 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f[62511]: 2024-04-25T13:39:21.082+0000 7fcd4c6d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:21.385 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:21 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f[62511]: 2024-04-25T13:39:21.082+0000 7fcd4c6d8700 -1 mds.f *** got signal Terminated *** 2024-04-25T13:39:25.357 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:25 smithi069 podman[69673]: 2024-04-25 13:39:25.090886319 +0000 UTC m=+4.118282301 container died b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T13:39:25.634 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:25 smithi069 podman[69673]: 2024-04-25 13:39:25.557460576 +0000 UTC m=+4.584856547 container cleanup b2048e2795f1e890d4e2ee15f7383b4fe941f2eeeb71faed15dabd466936de38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T13:39:25.635 INFO:journalctl@ceph.mds.f.smithi069.stdout:Apr 25 13:39:25 smithi069 bash[69673]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-f 2024-04-25T13:39:25.985 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T13:39:26.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:26.017 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T13:39:26.017 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T13:39:26.018 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.i 2024-04-25T13:39:26.335 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:26 smithi069 systemd[1]: Stopping Ceph mds.i for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:26.635 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: 2024-04-25T13:39:26.336+0000 7fe913809700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:26.635 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:26 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i[63123]: 2024-04-25T13:39:26.336+0000 7fe913809700 -1 mds.i *** got signal Terminated *** 2024-04-25T13:39:30.635 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:30 smithi069 podman[69796]: 2024-04-25 13:39:30.34761276 +0000 UTC m=+4.120388974 container died 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T13:39:31.135 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:30 smithi069 podman[69796]: 2024-04-25 13:39:30.93620644 +0000 UTC m=+4.708982656 container cleanup 54eeb09fb9c6867f67c8a713283e44c9e0ebdfd528567d4889012d6a85f1c228 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T13:39:31.135 INFO:journalctl@ceph.mds.i.smithi069.stdout:Apr 25 13:39:30 smithi069 bash[69796]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-i 2024-04-25T13:39:31.309 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T13:39:31.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:31.340 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T13:39:31.340 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T13:39:31.340 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.l 2024-04-25T13:39:31.884 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:31 smithi069 systemd[1]: Stopping Ceph mds.l for d766aeb0-0305-11ef-bc93-c7b262605968... 2024-04-25T13:39:31.885 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l[62186]: 2024-04-25T13:39:31.655+0000 7f37c3a21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T13:39:31.885 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:31 smithi069 ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l[62186]: 2024-04-25T13:39:31.655+0000 7f37c3a21700 -1 mds.l *** got signal Terminated *** 2024-04-25T13:39:36.065 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:35 smithi069 podman[69917]: 2024-04-25 13:39:35.662849967 +0000 UTC m=+4.106226220 container died fc1337416794d080da6985ded28dd42ea7641b4ea9ee4ad399be6cf675ba8f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-25T13:39:36.372 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:36 smithi069 podman[69917]: 2024-04-25 13:39:36.155782191 +0000 UTC m=+4.599158441 container cleanup fc1337416794d080da6985ded28dd42ea7641b4ea9ee4ad399be6cf675ba8f69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T13:39:36.372 INFO:journalctl@ceph.mds.l.smithi069.stdout:Apr 25 13:39:36 smithi069 bash[69917]: ceph-d766aeb0-0305-11ef-bc93-c7b262605968-mds-l 2024-04-25T13:39:36.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d766aeb0-0305-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T13:39:36.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T13:39:36.526 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T13:39:36.527 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:39:36.727 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:39.211 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:39:39.412 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:41.990 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T13:39:42.179 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:44.508 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:39:44.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:39:44.573 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T13:39:44.598 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T13:39:44.601 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi037/crash 2024-04-25T13:39:44.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:39:44.640 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:39:44.641 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:39:44.642 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi044/crash 2024-04-25T13:39:44.643 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:39:44.681 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:39:44.681 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:39:44.683 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi069/crash 2024-04-25T13:39:44.683 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash -- . 2024-04-25T13:39:44.717 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T13:39:44.717 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-04-25T13:39:44.718 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T13:39:44.718 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T13:39:44.756 INFO:tasks.cephadm:Compressing logs... 2024-04-25T13:39:44.756 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:39:44.799 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:39:44.801 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:39:44.823 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:39:44.823 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-04-25T13:39:44.825 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:39:44.825 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:39:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T13:39:44.826 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T13:39:44.837 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:39:44.837 INFO:teuthology.orchestra.run.smithi069.stderr:: No such file or directory 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr: --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T13:39:44.838 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr: 88.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log.gz/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.c.log: 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.z.log: /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr: 94.4% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 94.4% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr: --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T13:39:44.839 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.11.log: 83.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:39:44.840 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:39:44.840 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T13:39:44.840 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T13:39:44.843 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:39:44.846 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.x.log: 90.4% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:gzip 94.6% -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr: 84.1%/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.a.log: -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T13:39:44.847 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:39:44.848 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T13:39:44.848 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T13:39:44.848 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T13:39:44.849 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:39:44.849 INFO:teuthology.orchestra.run.smithi069.stderr: 91.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:39:44.849 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T13:39:44.850 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.1.log: 88.0%gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr: 90.5%/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.10.log: -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T13:39:44.851 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T13:39:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T13:39:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T13:39:44.852 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T13:39:44.854 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T13:39:44.860 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T13:39:44.897 INFO:teuthology.orchestra.run.smithi069.stderr: 94.1% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T13:39:44.957 INFO:teuthology.orchestra.run.smithi044.stderr: 93.5% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T13:39:45.102 INFO:teuthology.orchestra.run.smithi069.stderr: 92.1% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T13:39:45.182 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T13:39:45.230 INFO:teuthology.orchestra.run.smithi069.stderr: 92.7% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T13:39:45.231 INFO:teuthology.orchestra.run.smithi037.stderr: 92.8% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T13:39:45.235 INFO:teuthology.orchestra.run.smithi069.stderr: 92.7% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T13:39:45.255 INFO:teuthology.orchestra.run.smithi044.stderr: 92.1% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T13:39:45.279 INFO:teuthology.orchestra.run.smithi069.stderr: 93.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T13:39:45.313 INFO:teuthology.orchestra.run.smithi037.stderr: 93.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T13:39:45.315 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T13:39:45.336 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T13:39:45.340 INFO:teuthology.orchestra.run.smithi044.stderr: 92.6% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T13:39:45.354 INFO:teuthology.orchestra.run.smithi037.stderr: 92.7% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T13:39:45.402 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T13:39:45.404 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-25T13:39:45.404 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m0.587s 2024-04-25T13:39:45.404 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m2.590s 2024-04-25T13:39:45.404 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.103s 2024-04-25T13:39:45.424 INFO:teuthology.orchestra.run.smithi037.stderr: 93.0% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T13:39:45.443 INFO:teuthology.orchestra.run.smithi069.stderr: 92.8% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T13:39:45.445 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-04-25T13:39:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m0.630s 2024-04-25T13:39:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:user 0m2.192s 2024-04-25T13:39:45.445 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m0.098s 2024-04-25T13:39:45.703 INFO:teuthology.orchestra.run.smithi037.stderr: 90.5% -- replaced with /var/log/ceph/d766aeb0-0305-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T13:39:45.705 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T13:39:45.705 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m0.892s 2024-04-25T13:39:45.705 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m3.180s 2024-04-25T13:39:45.705 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.128s 2024-04-25T13:39:45.706 INFO:tasks.cephadm:Archiving logs... 2024-04-25T13:39:45.706 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi037/log 2024-04-25T13:39:45.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:39:46.185 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi044/log 2024-04-25T13:39:46.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:39:46.558 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi069/log 2024-04-25T13:39:46.559 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:39:46.914 INFO:tasks.cephadm:Removing cluster... 2024-04-25T13:39:46.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force 2024-04-25T13:39:47.112 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:47.385 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force 2024-04-25T13:39:47.586 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:47.845 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d766aeb0-0305-11ef-bc93-c7b262605968 --force 2024-04-25T13:39:48.023 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: d766aeb0-0305-11ef-bc93-c7b262605968 2024-04-25T13:39:48.280 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T13:39:48.280 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:39:48.296 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:39:48.313 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T13:39:48.328 INFO:tasks.cephadm:Teardown complete 2024-04-25T13:39:48.328 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:39:48.341 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:39:48.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:39:48.344 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:39:48.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:39:48.461 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:39:48.471 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:39:48.471 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:39:48.473 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:39:48.476 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:39:48.485 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:39:48.488 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:39:48.489 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-04-25T13:39:48.490 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:39:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T13:39:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +926us[ +929us] +/- 68ms 2024-04-25T13:39:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -1652us[-1652us] +/- 76ms 2024-04-25T13:39:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +711us[ +711us] +/- 84ms 2024-04-25T13:39:48.491 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +979us[ +948us] +/- 68ms 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -1648us[-1680us] +/- 76ms 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +731us[ +700us] +/- 84ms 2024-04-25T13:39:48.493 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +929us[ +926us] +/- 68ms 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -1649us[-1649us] +/- 76ms 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +711us[ +708us] +/- 84ms 2024-04-25T13:39:48.495 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T13:39:48.495 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:39:48.506 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:39:48.506 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:39:48.517 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:39:48.603 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:39:48.672 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T13:39:48.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:39:48.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:48.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:39:48.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:48.794 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T13:39:48.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:48.820 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:39:48.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:39:48.841 INFO:teuthology.task.internal:Duration was 1853.156015 seconds 2024-04-25T13:39:48.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:39:48.851 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:39:48.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:39:48.854 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:39:48.856 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:39:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:39:48.894 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:39:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T13:39:49.182 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:39:49.182 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T13:39:49.183 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:39:49.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-25T13:39:49.203 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:39:49.246 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-04-25T13:39:49.246 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:39:49.275 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:39:49.275 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:39:49.278 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:39:49.289 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:39:49.340 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:39:49.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:39:49.344 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:39:49.359 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:39:49.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:39:49.733 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:39:49.733 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:39:49.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:39:49.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:39:49.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:39:49.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.819 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.825 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.842 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-25T13:39:49.845 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-25T13:39:49.867 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-04-25T13:39:49.888 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:49.907 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:49.942 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:39:49.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:39:49.956 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:39:49.967 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:39:49.968 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi037 2024-04-25T13:39:49.968 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:39:50.008 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi044 2024-04-25T13:39:50.008 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:39:50.043 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466/remote/smithi069 2024-04-25T13:39:50.045 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:39:50.080 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:39:50.080 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:39:50.083 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:39:50.085 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:39:50.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:39:50.142 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:39:50.142 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:39:50.153 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:39:50.153 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:39:50.155 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:39:50.157 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:39:50.168 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:39 /home/ubuntu/cephtest 2024-04-25T13:39:50.170 INFO:teuthology.orchestra.run.smithi044.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:39 /home/ubuntu/cephtest 2024-04-25T13:39:50.182 INFO:teuthology.orchestra.run.smithi069.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:39 /home/ubuntu/cephtest 2024-04-25T13:39:50.183 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:39:50.195 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:39:50.360 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T13:39:50.385 DEBUG:teuthology.nuke:shortname: smithi037 2024-04-25T13:39:50.386 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:39:50.409 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.741619', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDPP72cLkBUWFsVOqjlr1j/ryPTLtkeju2zvuvEvZ7lNgcsXw3PRlD102Dda+ywF6itLI2Pe0pNKSbdAMAdRtEU='} 2024-04-25T13:39:50.413 DEBUG:teuthology.nuke:shortname: smithi044 2024-04-25T13:39:50.413 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:39:50.436 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.745324', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOD4lmyq3NdK2McDRLBKiZER1Oh3rB6xbnVAPzL1v8i19kftxVD3zAPtVE/9ulRz6OpkxASXTAp7lqTQWeasB3o='} 2024-04-25T13:39:50.441 DEBUG:teuthology.nuke:shortname: smithi069 2024-04-25T13:39:50.441 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:39:50.464 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_12:17:05-fs-wip-lusov-quiescer-distro-default-smithi/7673466', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:03:28.743586', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLv+CTxhdVUwfvxI7kfEa+OjaVy9jsj2qygh4fIrXpwYfmbNpahww1nQZaKIx7C5P5Pvm96OMskcdlHiKac+5Oc='} 2024-04-25T13:39:50.474 INFO:teuthology.orchestra.console.smithi037:Power off 2024-04-25T13:39:50.475 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:39:50.502 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-04-25T13:39:50.502 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:50.520 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-25T13:39:50.521 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:39:50.531 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-25T13:39:50.544 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-25T13:39:50.544 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:50.559 INFO:teuthology.orchestra.console.smithi069:Power off 2024-04-25T13:39:50.559 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T13:39:50.569 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-25T13:39:50.580 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-04-25T13:39:50.580 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:50.602 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-25T13:39:54.532 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:54.570 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:54.604 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:54.642 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-04-25T13:39:54.680 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-25T13:39:54.713 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-25T13:39:58.647 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:58.681 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:58.713 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:39:58.758 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-04-25T13:39:58.790 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-25T13:39:58.823 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-25T13:39:58.859 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-04-25T13:40:02.795 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:40:02.824 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T13:40:02.907 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-25T13:40:02.932 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-04-25T13:40:03.008 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-25T13:40:03.033 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-04-25T13:40:03.139 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 1853.156014919281 failure_reason: '''NoneType'' object has no attribute ''get''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=12044fe366324769b217fec27d767ecc status: fail success: false 2024-04-25T13:40:03.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:40:03.234 INFO:teuthology.run:FAIL